Title: An artificial intelligence model utilizing endoscopic ultrasonography for differentiating
small and micro gastric stromal tumors from gastric leiomyomas
Version: 0 Date: 09 Dec 2024
Reviewer's report:
General Comments
This study explores the development of an AI-assisted EUS system for the differential diagnosis
of gastric subepithelial lesions (SELs), particularly focusing on GISTs and GLs. While the
topic addresses a clinically relevant issue, significant overlap with recent studies and limitations
in methodology and focus raise questions about the study's novelty and impact. However,
this study’s unique emphasis on micro lesions (<10 mm) could represent a valuable niche
if adequately highlighted and developed. Substantial revisions are necessary to clarify its
objectives, improve methodological transparency, and establish its unique contribution.
Recent studies, including large-scale meta-analyses and multicenter trials, have demonstrated
the efficacy of AI in differentiating SELs, particularly GISTs and GLs, using EUS images. For
instance:
Dong et al. (2024)( DOI: 10.1016/j.eclinm.2024.102656) reported an AUC of 0.948 for AI-
assisted EUS in differentiating GISTs and GLs and included prospective evaluations.
Another meta-analysis highlighted AI's diagnostic accuracy for SELs across 44,154 images with
high sensitivity and specificity. (DOI: 10.4253/wjge.v15.i8.528)
Major Comments
1. Unclear Research Objectives
Issue: The primary objective of the study is not clearly defined. If the focus is solely on
evaluating the performance of the EUS-AI model, the inclusion of clinical feature-based models
introduces ambiguity.
Why were machine learning models incapable of sufficiently classifying GISTs and GLs
included as comparators?
Restricting the analysis to a comparison between the EUS-AI model and endoscopists' diagnostic
performance may simplify and clarify the study.
2. Inclusion of Target Cases
Issue: The study lacks information on the preoperative histological diagnosis of the included
cases. Specifically, the proportion of cases diagnosed through boring biopsy, EUS-FNAB, or
based solely on imaging findings is not mentioned.
Suggestion: If the goal is to develop a new solution for clinical decision-making based on optical
biopsy or imaging, preoperative diagnostic details should be incorporated. This will enhance the
relevance of the study to clinical practice.
3. CNN Models
Selection Criteria: The basis for selecting the three CNN models (DenseNet201, ResNet50,
VGG19) was not described. Were alternative models considered or tested? If not, justify why
these models were chosen.
Performance Metrics: Given the variability in datasets, including F1 scores alongside other
performance metrics (e.g., sensitivity, specificity, AUC) is recommended for a more balanced
evaluation. Hyperparameters: The methods section lacks details about hyperparameters used
during training (e.g., batch size, epochs, learning rate). This omission affects the study’s
reproducibility. Discussion: While ResNet50 was found to perform best, the reasons for its
superior performance were not adequately discussed.
4. Endoscopic Diagnosis
Diagnostic Criteria: The criteria used by endoscopists to differentiate GISTs and GLs were not
clearly defined. For instance, were specific patterns like mosaic patterns (indicative of GISTs)
or homogeneous patterns (indicative of GLs) employed? Vague diagnostic criteria undermine
the reliability of the findings. Intraobserver Agreement: It is suggested to re-evaluate the same
images at a later time and calculate intraobserver agreement (e.g., kappa values). If intraobserver
reliability is low, potential biases must be acknowledged. Unit of Analysis: The study does
not specify whether endoscopists diagnosed cases on a lesion-based or image-based basis. This
distinction is critical for fair comparisons with the EUS-AI model.
5. Discussion
Limitations: The discussion section should rigorously address the limitations of the study,
including: Single-center, retrospective design. Small sample size and single-race cohort, leading
to potential biases in clinical feature analysis. Restriction to tumors #2 cm, which limits
applicability to medium-sized SMTs (e.g., 2–5 cm). Exclusion of diverse SEL types such as
ectopic pancreas, schwannomas, granular cell tumors, and neuroendocrine tumors.
Title: An artificial intelligence model utilizing endoscopic ultrasonography for differentiating
small and micro gastric stromal tumors from gastric leiomyomas
Version: 0 Date: 09 Dec 2024
Reviewer's report:
General Comments
This study explores the development of an AI-assisted EUS system for the differential diagnosis
of gastric subepithelial lesions (SELs), particularly focusing on GISTs and GLs. While the
topic addresses a clinically relevant issue, significant overlap with recent studies and limitations
in methodology and focus raise questions about the study's novelty and impact. However,
this study’s unique emphasis on micro lesions (<10 mm) could represent a valuable niche
if adequately highlighted and developed. Substantial revisions are necessary to clarify its
objectives, improve methodological transparency, and establish its unique contribution.
Recent studies, including large-scale meta-analyses and multicenter trials, have demonstrated
the efficacy of AI in differentiating SELs, particularly GISTs and GLs, using EUS images. For
instance:
Dong et al. (2024)( DOI: 10.1016/j.eclinm.2024.102656) reported an AUC of 0.948 for AI-
assisted EUS in differentiating GISTs and GLs and included prospective evaluations.
Another meta-analysis highlighted AI's diagnostic accuracy for SELs across 44,154 images with
high sensitivity and specificity. (DOI: 10.4253/wjge.v15.i8.528)
Major Comments
1. Unclear Research Objectives
Issue: The primary objective of the study is not clearly defined. If the focus is solely on
evaluating the performance of the EUS-AI model, the inclusion of clinical feature-based models
introduces ambiguity.
Why were machine learning models incapable of sufficiently classifying GISTs and GLs
included as comparators?
Restricting the analysis to a comparison between the EUS-AI model and endoscopists' diagnostic
performance may simplify and clarify the study.
2. Inclusion of Target Cases
Issue: The study lacks information on the preoperative histological diagnosis of the included
cases. Specifically, the proportion of cases diagnosed through boring biopsy, EUS-FNAB, or
based solely on imaging findings is not mentioned.
Suggestion: If the goal is to develop a new solution for clinical decision-making based on optical
biopsy or imaging, preoperative diagnostic details should be incorporated. This will enhance the
relevance of the study to clinical practice.
3. CNN Models
Selection Criteria: The basis for selecting the three CNN models (DenseNet201, ResNet50,
VGG19) was not described. Were alternative models considered or tested? If not, justify why
these models were chosen.
Performance Metrics: Given the variability in datasets, including F1 scores alongside other
performance metrics (e.g., sensitivity, specificity, AUC) is recommended for a more balanced
evaluation. Hyperparameters: The methods section lacks details about hyperparameters used
during training (e.g., batch size, epochs, learning rate). This omission affects the study’s
reproducibility. Discussion: While ResNet50 was found to perform best, the reasons for its
superior performance were not adequately discussed.
4. Endoscopic Diagnosis
Diagnostic Criteria: The criteria used by endoscopists to differentiate GISTs and GLs were not
clearly defined. For instance, were specific patterns like mosaic patterns (indicative of GISTs)
or homogeneous patterns (indicative of GLs) employed? Vague diagnostic criteria undermine
the reliability of the findings. Intraobserver Agreement: It is suggested to re-evaluate the same
images at a later time and calculate intraobserver agreement (e.g., kappa values). If intraobserver
reliability is low, potential biases must be acknowledged. Unit of Analysis: The study does
not specify whether endoscopists diagnosed cases on a lesion-based or image-based basis. This
distinction is critical for fair comparisons with the EUS-AI model.
5. Discussion
Limitations: The discussion section should rigorously address the limitations of the study,
including: Single-center, retrospective design. Small sample size and single-race cohort, leading
to potential biases in clinical feature analysis. Restriction to tumors #2 cm, which limits
applicability to medium-sized SMTs (e.g., 2–5 cm). Exclusion of diverse SEL types such as
ectopic pancreas, schwannomas, granular cell tumors, and neuroendocrine tumors.
Please review this paper.
user
Title: An artificial intelligence model utilizing endoscopic ultrasonography for differentiating
small and micro gastric stromal tumors from gastric leiomyomas
Version: 0 Date: 09 Dec 2024
Reviewer's report:
General Comments
This study explores the development of an AI-assisted EUS system for the differential diagnosis
of gastric subepithelial lesions (SELs), particularly focusing on GISTs and GLs. While the
topic addresses a clinically relevant issue, significant overlap with recent studies and limitations
in methodology and focus raise questions about the study's novelty and impact. However,
this study’s unique emphasis on micro lesions (<10 mm) could represent a valuable niche
if adequately highlighted and developed. Substantial revisions are necessary to clarify its
objectives, improve methodological transparency, and establish its unique contribution.
Recent studies, including large-scale meta-analyses and multicenter trials, have demonstrated
the efficacy of AI in differentiating SELs, particularly GISTs and GLs, using EUS images. For
instance:
Dong et al. (2024)( DOI: 10.1016/j.eclinm.2024.102656) reported an AUC of 0.948 for AI-
assisted EUS in differentiating GISTs and GLs and included prospective evaluations.
Another meta-analysis highlighted AI's diagnostic accuracy for SELs across 44,154 images with
high sensitivity and specificity. (DOI: 10.4253/wjge.v15.i8.528)
Major Comments
1. Unclear Research Objectives
Issue: The primary objective of the study is not clearly defined. If the focus is solely on
evaluating the performance of the EUS-AI model, the inclusion of clinical feature-based models
introduces ambiguity.
Why were machine learning models incapable of sufficiently classifying GISTs and GLs
included as comparators?
Restricting the analysis to a comparison between the EUS-AI model and endoscopists' diagnostic
performance may simplify and clarify the study.
2. Inclusion of Target Cases
Issue: The study lacks information on the preoperative histological diagnosis of the included
cases. Specifically, the proportion of cases diagnosed through boring biopsy, EUS-FNAB, or
based solely on imaging findings is not mentioned.
Suggestion: If the goal is to develop a new solution for clinical decision-making based on optical
biopsy or imaging, preoperative diagnostic details should be incorporated. This will enhance the
relevance of the study to clinical practice.
3. CNN Models
Selection Criteria: The basis for selecting the three CNN models (DenseNet201, ResNet50,
VGG19) was not described. Were alternative models considered or tested? If not, justify why
these models were chosen.
Performance Metrics: Given the variability in datasets, including F1 scores alongside other
performance metrics (e.g., sensitivity, specificity, AUC) is recommended for a more balanced
evaluation. Hyperparameters: The methods section lacks details about hyperparameters used
during training (e.g., batch size, epochs, learning rate). This omission affects the study’s
reproducibility. Discussion: While ResNet50 was found to perform best, the reasons for its
superior performance were not adequately discussed.
4. Endoscopic Diagnosis
Diagnostic Criteria: The criteria used by endoscopists to differentiate GISTs and GLs were not
clearly defined. For instance, were specific patterns like mosaic patterns (indicative of GISTs)
or homogeneous patterns (indicative of GLs) employed? Vague diagnostic criteria undermine
the reliability of the findings. Intraobserver Agreement: It is suggested to re-evaluate the same
images at a later time and calculate intraobserver agreement (e.g., kappa values). If intraobserver
reliability is low, potential biases must be acknowledged. Unit of Analysis: The study does
not specify whether endoscopists diagnosed cases on a lesion-based or image-based basis. This
distinction is critical for fair comparisons with the EUS-AI model.
5. Discussion
Limitations: The discussion section should rigorously address the limitations of the study,
including: Single-center, retrospective design. Small sample size and single-race cohort, leading
to potential biases in clinical feature analysis. Restriction to tumors #2 cm, which limits
applicability to medium-sized SMTs (e.g., 2–5 cm). Exclusion of diverse SEL types such as
ectopic pancreas, schwannomas, granular cell tumors, and neuroendocrine tumors.
assistant