Title: Risk prediction and effect evaluation of complicated appendicitis based on XGBoost
modeling
Version: 0 Date: 05 Mar 2025
Reviewer's report:
The manuscript presents a machine learning model based on XGBoost for predicting
complicated appendicitis (CAP) and evaluating its clinical utility. The study is well-structured,
methodologically rigorous, and clinically relevant to gastroenterology. However, some key
concerns regarding methodology, validation, interpretability, and novelty need to be addressed
before it meets the publication standards of BMC Gastroenterology.
• The study is based on a single-center retrospective dataset (n=773). However, no details
are provided regarding potential selection biases, patient demographics, or generalizability.
The criteria for defining complicated vs. uncomplicated appendicitis are briefly mentioned but
should be explicitly defined in the methodology. The manuscript does not state how missing
values were addressed, which is critical in machine learning studies. Provide more details on
patient selection criteria, potential biases, missing data handling strategies, and generalizability
limitations.
• The Mean Decrease Accuracy function of Random Forest was used for feature selection, but
no comparison with other feature selection techniques (e.g., Boruta, LASSO) was performed.
Some selected features, such as muscular tone and WTOG24h (time from pain onset to surgery
>24 hours), require better justification for clinical relevance in CAP prediction. Justify why this
feature selection method was chosen, and provide clinical rationale for key predictors.
• The study evaluates model performance using AUC, accuracy, sensitivity, specificity,
precision, and F1-score, but confidence intervals (CIs) for these metrics are not provided,
making it difficult to assess statistical reliability. The Brier score is included for calibration
assessment, but additional metrics like log loss could improve model evaluation. Report CIs for
all performance metrics and consider additional calibration metrics.
• The XGBoost model performed extremely well (AUC = 0.996) in the training set but dropped
to 0.914 in the test set, raising concerns about potential overfitting. No external validation
dataset was used, limiting the model’s real-world applicability. Conduct external validation on
an independent dataset or apply cross-validation techniques to mitigate overfitting concerns.
• While important features are listed, the clinical rationale behind their importance is not well
discussed. The study lacks SHAP analysis or other explainability techniques, which would help
clinicians understand how features influence model predictions. Incorporate SHAP analysis or
detailed feature impact discussions to improve clinical interpretability.
• The manuscript includes DCA curves, but how this model should be integrated into clinical
workflows is unclear. Does this model provide actionable recommendations (e.g., immediate
surgery vs. conservative treatment)? Provide clear guidance on how clinicians should interpret
and use this model in practice.
• The manuscript briefly compares its model to other machine learning approaches for
appendicitis prediction, but it lacks a detailed discussion on how it improves upon existing
methods. Prior models using deep learning, CT imaging, or clinical scoring systems should be
critically reviewed. Strengthen the literature comparison and explicitly state what makes this
model novel.
The manuscript is promising but requires major revisions before it meets the publication
standards of BMC Gastroenterology. The authors should:
1. Enhance dataset transparency by detailing missing data handling and selection criteria.
2. Improve model validation by reporting confidence intervals and conducting external
validation.
3. Strengthen clinical interpretability using SHAP analysis and decision support frameworks.
4. Clarify novelty by comparing this model with prior approaches in appendicitis prediction.
Until these key issues are addressed, the manuscript is not yet suitable for publication.
Title: Risk prediction and effect evaluation of complicated appendicitis based on XGBoost
modeling
Version: 0 Date: 05 Mar 2025
Reviewer's report:
The manuscript presents a machine learning model based on XGBoost for predicting
complicated appendicitis (CAP) and evaluating its clinical utility. The study is well-structured,
methodologically rigorous, and clinically relevant to gastroenterology. However, some key
concerns regarding methodology, validation, interpretability, and novelty need to be addressed
before it meets the publication standards of BMC Gastroenterology.
• The study is based on a single-center retrospective dataset (n=773). However, no details
are provided regarding potential selection biases, patient demographics, or generalizability.
The criteria for defining complicated vs. uncomplicated appendicitis are briefly mentioned but
should be explicitly defined in the methodology. The manuscript does not state how missing
values were addressed, which is critical in machine learning studies. Provide more details on
patient selection criteria, potential biases, missing data handling strategies, and generalizability
limitations.
• The Mean Decrease Accuracy function of Random Forest was used for feature selection, but
no comparison with other feature selection techniques (e.g., Boruta, LASSO) was performed.
Some selected features, such as muscular tone and WTOG24h (time from pain onset to surgery
>24 hours), require better justification for clinical relevance in CAP prediction. Justify why this
feature selection method was chosen, and provide clinical rationale for key predictors.
• The study evaluates model performance using AUC, accuracy, sensitivity, specificity,
precision, and F1-score, but confidence intervals (CIs) for these metrics are not provided,
making it difficult to assess statistical reliability. The Brier score is included for calibration
assessment, but additional metrics like log loss could improve model evaluation. Report CIs for
all performance metrics and consider additional calibration metrics.
• The XGBoost model performed extremely well (AUC = 0.996) in the training set but dropped
to 0.914 in the test set, raising concerns about potential overfitting. No external validation
dataset was used, limiting the model’s real-world applicability. Conduct external validation on
an independent dataset or apply cross-validation techniques to mitigate overfitting concerns.
• While important features are listed, the clinical rationale behind their importance is not well
discussed. The study lacks SHAP analysis or other explainability techniques, which would help
clinicians understand how features influence model predictions. Incorporate SHAP analysis or
detailed feature impact discussions to improve clinical interpretability.
• The manuscript includes DCA curves, but how this model should be integrated into clinical
workflows is unclear. Does this model provide actionable recommendations (e.g., immediate
surgery vs. conservative treatment)? Provide clear guidance on how clinicians should interpret
and use this model in practice.
• The manuscript briefly compares its model to other machine learning approaches for
appendicitis prediction, but it lacks a detailed discussion on how it improves upon existing
methods. Prior models using deep learning, CT imaging, or clinical scoring systems should be
critically reviewed. Strengthen the literature comparison and explicitly state what makes this
model novel.
The manuscript is promising but requires major revisions before it meets the publication
standards of BMC Gastroenterology. The authors should:
1. Enhance dataset transparency by detailing missing data handling and selection criteria.
2. Improve model validation by reporting confidence intervals and conducting external
validation.
3. Strengthen clinical interpretability using SHAP analysis and decision support frameworks.
4. Clarify novelty by comparing this model with prior approaches in appendicitis prediction.
Until these key issues are addressed, the manuscript is not yet suitable for publication.
Please review this paper.
user
Title: Risk prediction and effect evaluation of complicated appendicitis based on XGBoost
modeling
Version: 0 Date: 05 Mar 2025
Reviewer's report:
The manuscript presents a machine learning model based on XGBoost for predicting
complicated appendicitis (CAP) and evaluating its clinical utility. The study is well-structured,
methodologically rigorous, and clinically relevant to gastroenterology. However, some key
concerns regarding methodology, validation, interpretability, and novelty need to be addressed
before it meets the publication standards of BMC Gastroenterology.
• The study is based on a single-center retrospective dataset (n=773). However, no details
are provided regarding potential selection biases, patient demographics, or generalizability.
The criteria for defining complicated vs. uncomplicated appendicitis are briefly mentioned but
should be explicitly defined in the methodology. The manuscript does not state how missing
values were addressed, which is critical in machine learning studies. Provide more details on
patient selection criteria, potential biases, missing data handling strategies, and generalizability
limitations.
• The Mean Decrease Accuracy function of Random Forest was used for feature selection, but
no comparison with other feature selection techniques (e.g., Boruta, LASSO) was performed.
Some selected features, such as muscular tone and WTOG24h (time from pain onset to surgery
>24 hours), require better justification for clinical relevance in CAP prediction. Justify why this
feature selection method was chosen, and provide clinical rationale for key predictors.
• The study evaluates model performance using AUC, accuracy, sensitivity, specificity,
precision, and F1-score, but confidence intervals (CIs) for these metrics are not provided,
making it difficult to assess statistical reliability. The Brier score is included for calibration
assessment, but additional metrics like log loss could improve model evaluation. Report CIs for
all performance metrics and consider additional calibration metrics.
• The XGBoost model performed extremely well (AUC = 0.996) in the training set but dropped
to 0.914 in the test set, raising concerns about potential overfitting. No external validation
dataset was used, limiting the model’s real-world applicability. Conduct external validation on
an independent dataset or apply cross-validation techniques to mitigate overfitting concerns.
• While important features are listed, the clinical rationale behind their importance is not well
discussed. The study lacks SHAP analysis or other explainability techniques, which would help
clinicians understand how features influence model predictions. Incorporate SHAP analysis or
detailed feature impact discussions to improve clinical interpretability.
• The manuscript includes DCA curves, but how this model should be integrated into clinical
workflows is unclear. Does this model provide actionable recommendations (e.g., immediate
surgery vs. conservative treatment)? Provide clear guidance on how clinicians should interpret
and use this model in practice.
• The manuscript briefly compares its model to other machine learning approaches for
appendicitis prediction, but it lacks a detailed discussion on how it improves upon existing
methods. Prior models using deep learning, CT imaging, or clinical scoring systems should be
critically reviewed. Strengthen the literature comparison and explicitly state what makes this
model novel.
The manuscript is promising but requires major revisions before it meets the publication
standards of BMC Gastroenterology. The authors should:
1. Enhance dataset transparency by detailing missing data handling and selection criteria.
2. Improve model validation by reporting confidence intervals and conducting external
validation.
3. Strengthen clinical interpretability using SHAP analysis and decision support frameworks.
4. Clarify novelty by comparing this model with prior approaches in appendicitis prediction.
Until these key issues are addressed, the manuscript is not yet suitable for publication.
assistant