Title: Explainable machine learning model for predicting acute pancreatitis mortality in the
intensive care unit
Version: 3 Date: 21 Jan 2025
Reviewer:  Anwar Ashraf Abouelnasr
Reviewer's report:
Dear Authors, please address the following points below
-While the authors mention that they analyzed and selected predictors based on statistical
and clinical significance, more information on the specific feature selection process would be
helpful. For example, which variables were most significant in predicting mortality, and how
were redundant or highly correlated variables handled? This would provide greater insight into
the clinical relevance of the chosen features.
-The authors mention that the data from the first 24 hours in ICU are used for vital signs
and laboratory results, but there could be more discussion on why this specific timeframe was
selected. For example, how does the timing of data collection impact the predictive power of
the model?
-While the inclusion of common comorbidities such as diabetes mellitus, hepatic cirrhosis, and
metastatic cancer is relevant, the choice of comorbidities may limit the generalizability of the
model. Including additional factors (e.g., chronic kidney disease or obesity) or more granular
comorbidity classifications could improve the model’s predictive accuracy. Further explanation
of the decision to include specific comorbidities would clarify how they impact the model.
-Although the authors mention that the model could identify low-risk patients for less intensive
monitoring, the paper could benefit from more discussion on how this ML model could be
integrated into real-world clinical practice. For example, would it be integrated with electronic
health record (EHR) systems to provide real-time risk assessments? How could clinicians easily
use and interpret the model’s predictions in daily practice?
-The focus on the ICU setting is appropriate, but some additional thoughts on how the model
could be used to inform triage decisions more broadly—such as in emergency departments or
during initial patient assessments—would provide further context for its potential impact.
-While the XGBoost model is compared against SOFA, APACHE IV, and BISAP, the paper
could provide more detail on the results of these comparisons. For example, how do the AUCs of
each model compare? Are there any notable differences in performance across patient subgroups
(e.g., age, severity of AP)? Additionally, a brief discussion of model calibration (e.g., using
Brier scores) would offer more insight into how well the model’s predicted probabilities align
with actual outcomes.
-The study’s focus on achieving a sensitivity of 100% for death prediction is a valid approach,
but it would be helpful to discuss the trade-offs between sensitivity and specificity in greater
detail. While high sensitivity is important for identifying high-risk patients, the impact of lower
specificity (e.g., false positives) should also be considered, particularly in terms of resource
allocation.
-Although the use of KNNImputer is an effective method for handling missing data, the authors
could provide more justification for this choice. For instance, how does KNNImputer compare to
other imputation techniques (e.g., mean imputation or multiple imputation by chained equations)
in terms of model performance?
-While the model achieves a sensitivity of 100% (very low risk identified as safe for non-
ICU monitoring), the specificity of 38% is a limitation. This means that a significant number
of low-risk patients are still being classified as needing ICU care, potentially leading to some
unnecessary admissions. The authors suggest that adding additional variables or using other
ML algorithms could further optimize the model, which is a good point. However, a deeper
exploration of how this trade-off between sensitivity and specificity impacts patient care would
be valuable.
-The study's reliance on achieving high sensitivity could potentially mask some of the trade-offs
in real-world applications. For example, could the specificity be improved without significantly
decreasing sensitivity? A more detailed discussion of these trade-offs would provide better
insight into how the model could be fine-tuned for clinical use.
-While the study focuses on ICU admissions, the application of this model in other settings, such
as the emergency department or general medical wards, could also be explored. Given that the
goal is to identify low-risk patients, this could improve triage decisions at multiple stages of
patient care. Exploring these potential uses of the model might help increase its overall impact.
Title: Explainable machine learning model for predicting acute pancreatitis mortality in the
intensive care unit
Version: 3 Date: 21 Jan 2025
Reviewer:  Anwar Ashraf Abouelnasr
Reviewer's report:
Dear Authors, please address the following points below
-While the authors mention that they analyzed and selected predictors based on statistical
and clinical significance, more information on the specific feature selection process would be
helpful. For example, which variables were most significant in predicting mortality, and how
were redundant or highly correlated variables handled? This would provide greater insight into
the clinical relevance of the chosen features.
-The authors mention that the data from the first 24 hours in ICU are used for vital signs
and laboratory results, but there could be more discussion on why this specific timeframe was
selected. For example, how does the timing of data collection impact the predictive power of
the model?
-While the inclusion of common comorbidities such as diabetes mellitus, hepatic cirrhosis, and
metastatic cancer is relevant, the choice of comorbidities may limit the generalizability of the
model. Including additional factors (e.g., chronic kidney disease or obesity) or more granular
comorbidity classifications could improve the model’s predictive accuracy. Further explanation
of the decision to include specific comorbidities would clarify how they impact the model.
-Although the authors mention that the model could identify low-risk patients for less intensive
monitoring, the paper could benefit from more discussion on how this ML model could be
integrated into real-world clinical practice. For example, would it be integrated with electronic
health record (EHR) systems to provide real-time risk assessments? How could clinicians easily
use and interpret the model’s predictions in daily practice?
-The focus on the ICU setting is appropriate, but some additional thoughts on how the model
could be used to inform triage decisions more broadly—such as in emergency departments or
during initial patient assessments—would provide further context for its potential impact.
-While the XGBoost model is compared against SOFA, APACHE IV, and BISAP, the paper
could provide more detail on the results of these comparisons. For example, how do the AUCs of
each model compare? Are there any notable differences in performance across patient subgroups
(e.g., age, severity of AP)? Additionally, a brief discussion of model calibration (e.g., using
Brier scores) would offer more insight into how well the model’s predicted probabilities align
with actual outcomes.
-The study’s focus on achieving a sensitivity of 100% for death prediction is a valid approach,
but it would be helpful to discuss the trade-offs between sensitivity and specificity in greater
detail. While high sensitivity is important for identifying high-risk patients, the impact of lower
specificity (e.g., false positives) should also be considered, particularly in terms of resource
allocation.
-Although the use of KNNImputer is an effective method for handling missing data, the authors
could provide more justification for this choice. For instance, how does KNNImputer compare to
other imputation techniques (e.g., mean imputation or multiple imputation by chained equations)
in terms of model performance?
-While the model achieves a sensitivity of 100% (very low risk identified as safe for non-
ICU monitoring), the specificity of 38% is a limitation. This means that a significant number
of low-risk patients are still being classified as needing ICU care, potentially leading to some
unnecessary admissions. The authors suggest that adding additional variables or using other
ML algorithms could further optimize the model, which is a good point. However, a deeper
exploration of how this trade-off between sensitivity and specificity impacts patient care would
be valuable.
-The study's reliance on achieving high sensitivity could potentially mask some of the trade-offs
in real-world applications. For example, could the specificity be improved without significantly
decreasing sensitivity? A more detailed discussion of these trade-offs would provide better
insight into how the model could be fine-tuned for clinical use.
-While the study focuses on ICU admissions, the application of this model in other settings, such
as the emergency department or general medical wards, could also be explored. Given that the
goal is to identify low-risk patients, this could improve triage decisions at multiple stages of
patient care. Exploring these potential uses of the model might help increase its overall impact.
Please review this paper.
user
Title: Explainable machine learning model for predicting acute pancreatitis mortality in the
intensive care unit
Version: 3 Date: 21 Jan 2025
Reviewer:  Anwar Ashraf Abouelnasr
Reviewer's report:
Dear Authors, please address the following points below
-While the authors mention that they analyzed and selected predictors based on statistical
and clinical significance, more information on the specific feature selection process would be
helpful. For example, which variables were most significant in predicting mortality, and how
were redundant or highly correlated variables handled? This would provide greater insight into
the clinical relevance of the chosen features.
-The authors mention that the data from the first 24 hours in ICU are used for vital signs
and laboratory results, but there could be more discussion on why this specific timeframe was
selected. For example, how does the timing of data collection impact the predictive power of
the model?
-While the inclusion of common comorbidities such as diabetes mellitus, hepatic cirrhosis, and
metastatic cancer is relevant, the choice of comorbidities may limit the generalizability of the
model. Including additional factors (e.g., chronic kidney disease or obesity) or more granular
comorbidity classifications could improve the model’s predictive accuracy. Further explanation
of the decision to include specific comorbidities would clarify how they impact the model.
-Although the authors mention that the model could identify low-risk patients for less intensive
monitoring, the paper could benefit from more discussion on how this ML model could be
integrated into real-world clinical practice. For example, would it be integrated with electronic
health record (EHR) systems to provide real-time risk assessments? How could clinicians easily
use and interpret the model’s predictions in daily practice?
-The focus on the ICU setting is appropriate, but some additional thoughts on how the model
could be used to inform triage decisions more broadly—such as in emergency departments or
during initial patient assessments—would provide further context for its potential impact.
-While the XGBoost model is compared against SOFA, APACHE IV, and BISAP, the paper
could provide more detail on the results of these comparisons. For example, how do the AUCs of
each model compare? Are there any notable differences in performance across patient subgroups
(e.g., age, severity of AP)? Additionally, a brief discussion of model calibration (e.g., using
Brier scores) would offer more insight into how well the model’s predicted probabilities align
with actual outcomes.
-The study’s focus on achieving a sensitivity of 100% for death prediction is a valid approach,
but it would be helpful to discuss the trade-offs between sensitivity and specificity in greater
detail. While high sensitivity is important for identifying high-risk patients, the impact of lower
specificity (e.g., false positives) should also be considered, particularly in terms of resource
allocation.
-Although the use of KNNImputer is an effective method for handling missing data, the authors
could provide more justification for this choice. For instance, how does KNNImputer compare to
other imputation techniques (e.g., mean imputation or multiple imputation by chained equations)
in terms of model performance?
-While the model achieves a sensitivity of 100% (very low risk identified as safe for non-
ICU monitoring), the specificity of 38% is a limitation. This means that a significant number
of low-risk patients are still being classified as needing ICU care, potentially leading to some
unnecessary admissions. The authors suggest that adding additional variables or using other
ML algorithms could further optimize the model, which is a good point. However, a deeper
exploration of how this trade-off between sensitivity and specificity impacts patient care would
be valuable.
-The study's reliance on achieving high sensitivity could potentially mask some of the trade-offs
in real-world applications. For example, could the specificity be improved without significantly
decreasing sensitivity? A more detailed discussion of these trade-offs would provide better
insight into how the model could be fine-tuned for clinical use.
-While the study focuses on ICU admissions, the application of this model in other settings, such
as the emergency department or general medical wards, could also be explored. Given that the
goal is to identify low-risk patients, this could improve triage decisions at multiple stages of
patient care. Exploring these potential uses of the model might help increase its overall impact.
assistant