Title: Machine Learning for Temporary Stoma After Intestinal Resection in Surgical Decision-
Making of Crohn's Disease
Version: 1 Date: 15 Aug 2024
Reviewer's report:
PEER REVIEWER ASSESSMENTS:
OBJECTIVE - Full research articles: is there a clear objective that addresses one or several
testable research questions? (Brief or other article types: is there a clear objective?)
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols) appropriate for
the objective?
No - there are minor issues
EXECUTION - Are the experiments and analyses performed with sufficient technical rigor to
allow confidence in the results?
No - there are major issues
STATISTICS - Is the use of statistics in the manuscript appropriate?
No - there are issues with the statistics in the study
INTERPRETATION - Is the current interpretation/discussion of the results reasonable and not
overstated?
No - there are major issues
OVERALL MANUSCRIPT POTENTIAL - Has the author addressed your concerns sufficiently
for you to now recommend the work as a technically sound contribution? If not, can further
revisions be made to make the work technically sound?
No - manuscript has some fundamental flaw(s)
PEER REVIEWER COMMENTS:
GENERAL COMMENTS: I thank the authors for providing a detailed response to my
previous comments. However, the major concerns raised in my earlier reviews have not
been sufficiently addressed. Overall, the study report did not follow the TRIPOD guidelines,
lacking essential details in the methods and results sections. This omission undermines the
transparency, reproducibility, and validity of the findings. I recommend that the authors
refer to the TRIPOD Checklist for Prediction Model Development and Validation (available
at https://www.tripod-statement.org/wp-content/uploads/2020/01/Tripod-Checklist-Prediction-
Model-Development-and-Validation-PDF.pdf) for guidance. Additionally, two major issues
with the methods remain, which I have outlined below.
REQUESTED REVISIONS:
1. Stepwise regression has significant drawbacks for variable selection, as it can lead
to overfitting, where the model captures noise rather than the true signal, reducing the
generalizability of the results. The method is also unstable, with small changes in data
potentially leading to dramatic alterations in the selected variables, and it can produce biased
coefficient estimates due to repeated testing. Furthermore, stepwise regression often overlooks
multicollinearity, resulting in the inclusion of redundant variables, and lacks theoretical
guidance, which may lead to the selection of variables that are not meaningful from a domain
perspective. A more robust approach is to use regularization methods like Lasso or Ridge
regression, which apply penalties to prevent overfitting and manage multicollinearity. Elastic
Net, which combines the strengths of both Lasso and Ridge, is particularly effective when
dealing with correlated predictors. Additionally, information criterion-based methods like AIC
or BIC, Bayesian Model Averaging, and cross-validation techniques provide better alternatives
by balancing model fit and complexity, ensuring a more reliable variable selection process. The
authors should consider incorporating these state-of-the-art methods in their prediction studies.
2. While randomly splitting data into training and validation sets can give an initial sense of
model performance on unseen data, this approach has significant limitations compared to cross-
validation. A single random split may not capture the full variability in the data, potentially
leading to an overestimation or underestimation of model performance depending on how the
split was done. This can result in a model that either overfits or underfits the data. Cross-
validation, particularly k-fold cross-validation, offers a more robust assessment by repeatedly
splitting the data into different training and validation sets, ensuring that every data point
is used for both purposes across multiple iterations. This approach minimizes the risk of
relying on an unrepresentative random split and provides a more reliable estimate of model
performance, especially in small or imbalanced datasets. Cross-validation (or internal validation,
as recommended in TRIPOD checklist item 10b) is a routine and efficient procedure for model
training in prediction studies. It is regrettable that the authors did not follow this recommended
practice.
Title: Machine Learning for Temporary Stoma After Intestinal Resection in Surgical Decision-
Making of Crohn's Disease
Version: 1 Date: 15 Aug 2024
Reviewer's report:
PEER REVIEWER ASSESSMENTS:
OBJECTIVE - Full research articles: is there a clear objective that addresses one or several
testable research questions? (Brief or other article types: is there a clear objective?)
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols) appropriate for
the objective?
No - there are minor issues
EXECUTION - Are the experiments and analyses performed with sufficient technical rigor to
allow confidence in the results?
No - there are major issues
STATISTICS - Is the use of statistics in the manuscript appropriate?
No - there are issues with the statistics in the study
INTERPRETATION - Is the current interpretation/discussion of the results reasonable and not
overstated?
No - there are major issues
OVERALL MANUSCRIPT POTENTIAL - Has the author addressed your concerns sufficiently
for you to now recommend the work as a technically sound contribution? If not, can further
revisions be made to make the work technically sound?
No - manuscript has some fundamental flaw(s)
PEER REVIEWER COMMENTS:
GENERAL COMMENTS: I thank the authors for providing a detailed response to my
previous comments. However, the major concerns raised in my earlier reviews have not
been sufficiently addressed. Overall, the study report did not follow the TRIPOD guidelines,
lacking essential details in the methods and results sections. This omission undermines the
transparency, reproducibility, and validity of the findings. I recommend that the authors
refer to the TRIPOD Checklist for Prediction Model Development and Validation (available
at https://www.tripod-statement.org/wp-content/uploads/2020/01/Tripod-Checklist-Prediction-
Model-Development-and-Validation-PDF.pdf) for guidance. Additionally, two major issues
with the methods remain, which I have outlined below.
REQUESTED REVISIONS:
1. Stepwise regression has significant drawbacks for variable selection, as it can lead
to overfitting, where the model captures noise rather than the true signal, reducing the
generalizability of the results. The method is also unstable, with small changes in data
potentially leading to dramatic alterations in the selected variables, and it can produce biased
coefficient estimates due to repeated testing. Furthermore, stepwise regression often overlooks
multicollinearity, resulting in the inclusion of redundant variables, and lacks theoretical
guidance, which may lead to the selection of variables that are not meaningful from a domain
perspective. A more robust approach is to use regularization methods like Lasso or Ridge
regression, which apply penalties to prevent overfitting and manage multicollinearity. Elastic
Net, which combines the strengths of both Lasso and Ridge, is particularly effective when
dealing with correlated predictors. Additionally, information criterion-based methods like AIC
or BIC, Bayesian Model Averaging, and cross-validation techniques provide better alternatives
by balancing model fit and complexity, ensuring a more reliable variable selection process. The
authors should consider incorporating these state-of-the-art methods in their prediction studies.
2. While randomly splitting data into training and validation sets can give an initial sense of
model performance on unseen data, this approach has significant limitations compared to cross-
validation. A single random split may not capture the full variability in the data, potentially
leading to an overestimation or underestimation of model performance depending on how the
split was done. This can result in a model that either overfits or underfits the data. Cross-
validation, particularly k-fold cross-validation, offers a more robust assessment by repeatedly
splitting the data into different training and validation sets, ensuring that every data point
is used for both purposes across multiple iterations. This approach minimizes the risk of
relying on an unrepresentative random split and provides a more reliable estimate of model
performance, especially in small or imbalanced datasets. Cross-validation (or internal validation,
as recommended in TRIPOD checklist item 10b) is a routine and efficient procedure for model
training in prediction studies. It is regrettable that the authors did not follow this recommended
practice.
Please review this paper.
user
Title: Machine Learning for Temporary Stoma After Intestinal Resection in Surgical Decision-
Making of Crohn's Disease
Version: 1 Date: 15 Aug 2024
Reviewer's report:
PEER REVIEWER ASSESSMENTS:
OBJECTIVE - Full research articles: is there a clear objective that addresses one or several
testable research questions? (Brief or other article types: is there a clear objective?)
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols) appropriate for
the objective?
No - there are minor issues
EXECUTION - Are the experiments and analyses performed with sufficient technical rigor to
allow confidence in the results?
No - there are major issues
STATISTICS - Is the use of statistics in the manuscript appropriate?
No - there are issues with the statistics in the study
INTERPRETATION - Is the current interpretation/discussion of the results reasonable and not
overstated?
No - there are major issues
OVERALL MANUSCRIPT POTENTIAL - Has the author addressed your concerns sufficiently
for you to now recommend the work as a technically sound contribution? If not, can further
revisions be made to make the work technically sound?
No - manuscript has some fundamental flaw(s)
PEER REVIEWER COMMENTS:
GENERAL COMMENTS: I thank the authors for providing a detailed response to my
previous comments. However, the major concerns raised in my earlier reviews have not
been sufficiently addressed. Overall, the study report did not follow the TRIPOD guidelines,
lacking essential details in the methods and results sections. This omission undermines the
transparency, reproducibility, and validity of the findings. I recommend that the authors
refer to the TRIPOD Checklist for Prediction Model Development and Validation (available
at https://www.tripod-statement.org/wp-content/uploads/2020/01/Tripod-Checklist-Prediction-
Model-Development-and-Validation-PDF.pdf) for guidance. Additionally, two major issues
with the methods remain, which I have outlined below.
REQUESTED REVISIONS:
1. Stepwise regression has significant drawbacks for variable selection, as it can lead
to overfitting, where the model captures noise rather than the true signal, reducing the
generalizability of the results. The method is also unstable, with small changes in data
potentially leading to dramatic alterations in the selected variables, and it can produce biased
coefficient estimates due to repeated testing. Furthermore, stepwise regression often overlooks
multicollinearity, resulting in the inclusion of redundant variables, and lacks theoretical
guidance, which may lead to the selection of variables that are not meaningful from a domain
perspective. A more robust approach is to use regularization methods like Lasso or Ridge
regression, which apply penalties to prevent overfitting and manage multicollinearity. Elastic
Net, which combines the strengths of both Lasso and Ridge, is particularly effective when
dealing with correlated predictors. Additionally, information criterion-based methods like AIC
or BIC, Bayesian Model Averaging, and cross-validation techniques provide better alternatives
by balancing model fit and complexity, ensuring a more reliable variable selection process. The
authors should consider incorporating these state-of-the-art methods in their prediction studies.
2. While randomly splitting data into training and validation sets can give an initial sense of
model performance on unseen data, this approach has significant limitations compared to cross-
validation. A single random split may not capture the full variability in the data, potentially
leading to an overestimation or underestimation of model performance depending on how the
split was done. This can result in a model that either overfits or underfits the data. Cross-
validation, particularly k-fold cross-validation, offers a more robust assessment by repeatedly
splitting the data into different training and validation sets, ensuring that every data point
is used for both purposes across multiple iterations. This approach minimizes the risk of
relying on an unrepresentative random split and provides a more reliable estimate of model
performance, especially in small or imbalanced datasets. Cross-validation (or internal validation,
as recommended in TRIPOD checklist item 10b) is a routine and efficient procedure for model
training in prediction studies. It is regrettable that the authors did not follow this recommended
practice.
assistant