Title: Analyzing Risk Factors and Constructing a Predictive Model for Superficial Esophageal
Carcinoma with Submucosal Infiltration Exceeding 200 Micrometers.
Version: 3 Date: 16 Sep 2024
Reviewer's report:
This study aimed to identify risk factors for early-stage esophageal cancer that invades the
submucosal layer at a depth of 200 micrometers and to develop and validate a predictive
model. The study found that a logistic regression model complemented by SHAP visualizations
effectively identifies early esophageal cancer reaching 200 micrometers into the submucosa. The
study presents a well-structured methodology with a strong foundation in retrospective cohort
analysis and machine learning (ML) model application.
Comments
1. The authors used K-Nearest Neighbors (KNN) imputation for missing data, which is generally
reliable but can introduce bias if the missing data mechanism is not random. The authors need
to ensure that the rationale for choosing KNN over other imputation methods is explained, as
this can affect the robustness of the model.
2. Although the model performed well in the authors' internal validation (AUCs of 0.899 and
0.850 for validation and test sets, respectively), it would be helpful to include external validation
or cross-institutional data to generalize findings across different patient populations.
3. The transformation of continuous variables into categorical ones using ROC curves (e.g.,
dichotomizing age at 71 years) should be justified. ROC curve thresholds may not always reflect
clinically meaningful cutoffs and could oversimplify nuanced data.
4. The large number of models tested (XGBoost, Logistic Regression, LightGBM, etc.) and the
relatively small validation set (n=122) raise concerns about overfitting. Although the authors
performed cross-validation, it may be beneficial to emphasize steps taken to avoid overfitting,
such as using an independent test set or more rigorous cross-validation techniques.
5. There are far too many external figures that are difficult to read, there is also a Table with
far too much information, plus an Excel file that I don't seem to understand its purpose. Please
authors, also provide us with more detailed explanations of Figures 4c, 4d, and 4e to help readers
better interpret the differences in SHAP values and prediction errors. Do make our lives easier
when you present us your data!
6. While Logistic Regression is deemed the optimal model for clinical application based on
its performance, the authors note that RandomForest and XGBoost performed better in the
training set. A clearer rationale as to why Logistic Regression was favored despite the higher
performance of these models would enhance the discussion.
7. While the authors mention the variability in ESD guidelines, it would be useful to elaborate
more on how these variances affect clinical decision-making. Are there specific areas where
guidelines diverge significantly (e.g., eligibility for ESD vs. surgery)? This could provide more
context for why the predictive model that the study proposes would be useful in standardizing
treatment decisions.
8. The authors state that 6.2% of patients required surgical intervention post-endoscopy due to
unexpected SM1 infiltration. Although this is a relatively low percentage, it could be helpful to
compare this rate to similar studies. This would give the reader a better understanding of whether
this figure reflects a major clinical challenge or falls within the expected range.
9. The authors declare that logistic regression performed best among the machine learning
models tested. However, it would strengthen their argument if they provided more justification
for selecting logistic regression over more advanced models like XGBoost or Random Forest,
which tend to handle complex data better. Did logistic regression perform better across all
metrics (AUC, PR, DCA)? Or was it selected for its simplicity and clinical interpretability?
Clarifying this would help justify the model choice.
10. The section in the Discussion on the inflammatory markers (NLR, PLR, etc.) could benefit
from a clearer discussion on how PLR specifically influences tumor invasion. While the authors
briefly mention tumor progression, immune evasion, and angiogenesis, expanding this part to
discuss how these biological mechanisms specifically relate to the depth of esophageal cancer
invasion could make the argument more robust. Furthermore, acknowledging the need for
additional validation of inflammatory markers in larger cohorts could make this section more
balanced.
11. In the Discussion, it might be useful to briefly suggest potential solutions, such as
integrating wearable health monitoring devices or using biochemical markers (e.g., for smoking
or alcohol consumption) in future studies. This would demonstrate a forward-thinking approach
to addressing limitations.
12. The discussion of pickled food and salted meat consumption as risk factors is interesting,
but the authors could expand on the mechanisms behind the link to esophageal carcinoma. For
instance, the authors mention N-nitroso compounds, but a deeper discussion of the mechanisms
involved in the carcinogenesis could strengthen this argument. Additionally, consider discussing
whether this finding has implications for dietary recommendations in high-risk regions.
13. The single-center, retrospective nature of the study is noted as a limitation, but the authors
should discuss more thoroughly how generalizable their findings are to other populations,
especially outside high-risk regions (e.g., East Asia). Including plans for prospective, multi-
center studies would help address concerns about external validity.
14. While the authors underscore that the model can help clinicians in decision-making, a
clearer explanation of how this tool can be integrated into current clinical workflows would be
useful. Will this predictive model be part of an endoscopy software package, or is it something
that clinicians would use independently? Detailing its potential application could help readers
understand its practical utility.
15. The authors mention potential errors introduced by random partitioning of their data. This
could be expanded by discussing alternative partitioning methods, such as stratified sampling,
to mitigate this issue in future research. Additionally, emphasizing how their 5-fold cross-
validation helped reduce overfitting risks might reassure readers about the model’s reliability.
Title: Analyzing Risk Factors and Constructing a Predictive Model for Superficial Esophageal
Carcinoma with Submucosal Infiltration Exceeding 200 Micrometers.
Version: 3 Date: 16 Sep 2024
Reviewer's report:
This study aimed to identify risk factors for early-stage esophageal cancer that invades the
submucosal layer at a depth of 200 micrometers and to develop and validate a predictive
model. The study found that a logistic regression model complemented by SHAP visualizations
effectively identifies early esophageal cancer reaching 200 micrometers into the submucosa. The
study presents a well-structured methodology with a strong foundation in retrospective cohort
analysis and machine learning (ML) model application.
Comments
1. The authors used K-Nearest Neighbors (KNN) imputation for missing data, which is generally
reliable but can introduce bias if the missing data mechanism is not random. The authors need
to ensure that the rationale for choosing KNN over other imputation methods is explained, as
this can affect the robustness of the model.
2. Although the model performed well in the authors' internal validation (AUCs of 0.899 and
0.850 for validation and test sets, respectively), it would be helpful to include external validation
or cross-institutional data to generalize findings across different patient populations.
3. The transformation of continuous variables into categorical ones using ROC curves (e.g.,
dichotomizing age at 71 years) should be justified. ROC curve thresholds may not always reflect
clinically meaningful cutoffs and could oversimplify nuanced data.
4. The large number of models tested (XGBoost, Logistic Regression, LightGBM, etc.) and the
relatively small validation set (n=122) raise concerns about overfitting. Although the authors
performed cross-validation, it may be beneficial to emphasize steps taken to avoid overfitting,
such as using an independent test set or more rigorous cross-validation techniques.
5. There are far too many external figures that are difficult to read, there is also a Table with
far too much information, plus an Excel file that I don't seem to understand its purpose. Please
authors, also provide us with more detailed explanations of Figures 4c, 4d, and 4e to help readers
better interpret the differences in SHAP values and prediction errors. Do make our lives easier
when you present us your data!
6. While Logistic Regression is deemed the optimal model for clinical application based on
its performance, the authors note that RandomForest and XGBoost performed better in the
training set. A clearer rationale as to why Logistic Regression was favored despite the higher
performance of these models would enhance the discussion.
7. While the authors mention the variability in ESD guidelines, it would be useful to elaborate
more on how these variances affect clinical decision-making. Are there specific areas where
guidelines diverge significantly (e.g., eligibility for ESD vs. surgery)? This could provide more
context for why the predictive model that the study proposes would be useful in standardizing
treatment decisions.
8. The authors state that 6.2% of patients required surgical intervention post-endoscopy due to
unexpected SM1 infiltration. Although this is a relatively low percentage, it could be helpful to
compare this rate to similar studies. This would give the reader a better understanding of whether
this figure reflects a major clinical challenge or falls within the expected range.
9. The authors declare that logistic regression performed best among the machine learning
models tested. However, it would strengthen their argument if they provided more justification
for selecting logistic regression over more advanced models like XGBoost or Random Forest,
which tend to handle complex data better. Did logistic regression perform better across all
metrics (AUC, PR, DCA)? Or was it selected for its simplicity and clinical interpretability?
Clarifying this would help justify the model choice.
10. The section in the Discussion on the inflammatory markers (NLR, PLR, etc.) could benefit
from a clearer discussion on how PLR specifically influences tumor invasion. While the authors
briefly mention tumor progression, immune evasion, and angiogenesis, expanding this part to
discuss how these biological mechanisms specifically relate to the depth of esophageal cancer
invasion could make the argument more robust. Furthermore, acknowledging the need for
additional validation of inflammatory markers in larger cohorts could make this section more
balanced.
11. In the Discussion, it might be useful to briefly suggest potential solutions, such as
integrating wearable health monitoring devices or using biochemical markers (e.g., for smoking
or alcohol consumption) in future studies. This would demonstrate a forward-thinking approach
to addressing limitations.
12. The discussion of pickled food and salted meat consumption as risk factors is interesting,
but the authors could expand on the mechanisms behind the link to esophageal carcinoma. For
instance, the authors mention N-nitroso compounds, but a deeper discussion of the mechanisms
involved in the carcinogenesis could strengthen this argument. Additionally, consider discussing
whether this finding has implications for dietary recommendations in high-risk regions.
13. The single-center, retrospective nature of the study is noted as a limitation, but the authors
should discuss more thoroughly how generalizable their findings are to other populations,
especially outside high-risk regions (e.g., East Asia). Including plans for prospective, multi-
center studies would help address concerns about external validity.
14. While the authors underscore that the model can help clinicians in decision-making, a
clearer explanation of how this tool can be integrated into current clinical workflows would be
useful. Will this predictive model be part of an endoscopy software package, or is it something
that clinicians would use independently? Detailing its potential application could help readers
understand its practical utility.
15. The authors mention potential errors introduced by random partitioning of their data. This
could be expanded by discussing alternative partitioning methods, such as stratified sampling,
to mitigate this issue in future research. Additionally, emphasizing how their 5-fold cross-
validation helped reduce overfitting risks might reassure readers about the model’s reliability.
Please review this paper.
user
Title: Analyzing Risk Factors and Constructing a Predictive Model for Superficial Esophageal
Carcinoma with Submucosal Infiltration Exceeding 200 Micrometers.
Version: 3 Date: 16 Sep 2024
Reviewer's report:
This study aimed to identify risk factors for early-stage esophageal cancer that invades the
submucosal layer at a depth of 200 micrometers and to develop and validate a predictive
model. The study found that a logistic regression model complemented by SHAP visualizations
effectively identifies early esophageal cancer reaching 200 micrometers into the submucosa. The
study presents a well-structured methodology with a strong foundation in retrospective cohort
analysis and machine learning (ML) model application.
Comments
1. The authors used K-Nearest Neighbors (KNN) imputation for missing data, which is generally
reliable but can introduce bias if the missing data mechanism is not random. The authors need
to ensure that the rationale for choosing KNN over other imputation methods is explained, as
this can affect the robustness of the model.
2. Although the model performed well in the authors' internal validation (AUCs of 0.899 and
0.850 for validation and test sets, respectively), it would be helpful to include external validation
or cross-institutional data to generalize findings across different patient populations.
3. The transformation of continuous variables into categorical ones using ROC curves (e.g.,
dichotomizing age at 71 years) should be justified. ROC curve thresholds may not always reflect
clinically meaningful cutoffs and could oversimplify nuanced data.
4. The large number of models tested (XGBoost, Logistic Regression, LightGBM, etc.) and the
relatively small validation set (n=122) raise concerns about overfitting. Although the authors
performed cross-validation, it may be beneficial to emphasize steps taken to avoid overfitting,
such as using an independent test set or more rigorous cross-validation techniques.
5. There are far too many external figures that are difficult to read, there is also a Table with
far too much information, plus an Excel file that I don't seem to understand its purpose. Please
authors, also provide us with more detailed explanations of Figures 4c, 4d, and 4e to help readers
better interpret the differences in SHAP values and prediction errors. Do make our lives easier
when you present us your data!
6. While Logistic Regression is deemed the optimal model for clinical application based on
its performance, the authors note that RandomForest and XGBoost performed better in the
training set. A clearer rationale as to why Logistic Regression was favored despite the higher
performance of these models would enhance the discussion.
7. While the authors mention the variability in ESD guidelines, it would be useful to elaborate
more on how these variances affect clinical decision-making. Are there specific areas where
guidelines diverge significantly (e.g., eligibility for ESD vs. surgery)? This could provide more
context for why the predictive model that the study proposes would be useful in standardizing
treatment decisions.
8. The authors state that 6.2% of patients required surgical intervention post-endoscopy due to
unexpected SM1 infiltration. Although this is a relatively low percentage, it could be helpful to
compare this rate to similar studies. This would give the reader a better understanding of whether
this figure reflects a major clinical challenge or falls within the expected range.
9. The authors declare that logistic regression performed best among the machine learning
models tested. However, it would strengthen their argument if they provided more justification
for selecting logistic regression over more advanced models like XGBoost or Random Forest,
which tend to handle complex data better. Did logistic regression perform better across all
metrics (AUC, PR, DCA)? Or was it selected for its simplicity and clinical interpretability?
Clarifying this would help justify the model choice.
10. The section in the Discussion on the inflammatory markers (NLR, PLR, etc.) could benefit
from a clearer discussion on how PLR specifically influences tumor invasion. While the authors
briefly mention tumor progression, immune evasion, and angiogenesis, expanding this part to
discuss how these biological mechanisms specifically relate to the depth of esophageal cancer
invasion could make the argument more robust. Furthermore, acknowledging the need for
additional validation of inflammatory markers in larger cohorts could make this section more
balanced.
11. In the Discussion, it might be useful to briefly suggest potential solutions, such as
integrating wearable health monitoring devices or using biochemical markers (e.g., for smoking
or alcohol consumption) in future studies. This would demonstrate a forward-thinking approach
to addressing limitations.
12. The discussion of pickled food and salted meat consumption as risk factors is interesting,
but the authors could expand on the mechanisms behind the link to esophageal carcinoma. For
instance, the authors mention N-nitroso compounds, but a deeper discussion of the mechanisms
involved in the carcinogenesis could strengthen this argument. Additionally, consider discussing
whether this finding has implications for dietary recommendations in high-risk regions.
13. The single-center, retrospective nature of the study is noted as a limitation, but the authors
should discuss more thoroughly how generalizable their findings are to other populations,
especially outside high-risk regions (e.g., East Asia). Including plans for prospective, multi-
center studies would help address concerns about external validity.
14. While the authors underscore that the model can help clinicians in decision-making, a
clearer explanation of how this tool can be integrated into current clinical workflows would be
useful. Will this predictive model be part of an endoscopy software package, or is it something
that clinicians would use independently? Detailing its potential application could help readers
understand its practical utility.
15. The authors mention potential errors introduced by random partitioning of their data. This
could be expanded by discussing alternative partitioning methods, such as stratified sampling,
to mitigate this issue in future research. Additionally, emphasizing how their 5-fold cross-
validation helped reduce overfitting risks might reassure readers about the model’s reliability.
assistant