Title: Development and validation of an interpretable machine learning model for predicting
the risk of hepatocellular carcinoma in patients with chronic hepatitis B: a case-control study
Version: 1 Date: 22 Nov 2024
Reviewer's report:
PEER REVIEWER ASSESSMENTS:
OBJECTIVE - Full research articles: is there a clear objective that addresses a testable research
question(s) (brief or other article types: is there a clear objective)?
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols) appropriate for
the objective?
No - there are minor issues
EXECUTION - Are the experiments and analyses performed with technical rigor to allow
confidence in the results?
No - there are minor issues
STATISTICS - Is the use of statistics in the manuscript appropriate?
No - there are issues with the statistics in the study
INTERPRETATION - Is the current interpretation/discussion of the results reasonable and not
overstated?
No - there are major issues
OVERALL MANUSCRIPT POTENTIAL - Is the current version of this work technically
sound? If not, can revisions be made to make the work technically sound?
Maybe - with major revisions
PEER REVIEWER COMMENTS:
GENERAL COMMENTS: The manuscript investigated an approach to predict hepatocellular
carcinoma (HCC) risk using artificial neural networks (ANNs) based on routine clinical and
demographic data from patients with chronic hepatitis B. It demonstrates a fine methodological
framework with comprehensive variable selection techniques and robust model evaluation
metrics. However, significant limitations impact its rigor and generalizability. The reliance on
data from a single center, combined with the absence of explicit training-test data splitting, raises
concerns about potential overfitting and bias. The small sample size further limits the model's
external validity. While the discussion effectively highlights the model's clinical relevance, it
underemphasizes critical limitations such as the lack of both internal and external validation
and geographic diversity. Overall, the study is a promising contribution to HCC prediction but
requires methodological refinement and broader validation to enhance its clinical applicability.
I provide detailed comments below for further improvement.
REQUESTED REVISIONS:
1. The manuscript briefly mentions the inclusion and exclusion criteria but does not specify
whether there were missing values in the final dataset or how missing or incomplete data were
handled. Additionally, no flowchart detailing participant recruitment is provided, which would
improve clarity and transparency.
2. While the manuscript claims no collinearity among variables, it lacks explicit details about
the diagnostics used to confirm this (e.g., variance inflation factors).
3. The study constructs five ANN models using different sets of predictors selected by various
variable selection methods. However, the authors do not compare the ANN models with simpler
approaches such as logistic regression, random forest (RF), support vector machines (SVM),
regularized regression, or best subset regression. Given the small sample size and limited number
of predictors, simpler models might perform comparably to the more complex ANN models.
4. The manuscript describes the construction of ANN models and their evaluation using
metrics such as AUC, calibration curves, and the Hosmer–Lemeshow test. However, it does not
explicitly describe whether the data was split into training and testing datasets. Training-test
splitting is essential to prevent overfitting and provide reliable performance metrics. Without
clear evidence of this step, the risk of overfitting is particularly high, especially for ANN models,
which are sensitive to noise in the data.
5. The discussion suggests that Model 5 is ready for clinical application without addressing the
inherent limitations of ANN models, such as potential overfitting and dependency on specific
sample characteristics.
6. Claims about the superiority of the ANN approach over traditional methods are not
substantiated by comparative analyses in the current study.
7. I recommend that the authors thoroughly revise the manuscript following the TRIPOD
guidelines for AI-based predictive models and include a completed TRIPOD checklist upon
resubmission.
Title: Development and validation of an interpretable machine learning model for predicting
the risk of hepatocellular carcinoma in patients with chronic hepatitis B: a case-control study
Version: 1 Date: 22 Nov 2024
Reviewer's report:
PEER REVIEWER ASSESSMENTS:
OBJECTIVE - Full research articles: is there a clear objective that addresses a testable research
question(s) (brief or other article types: is there a clear objective)?
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols) appropriate for
the objective?
No - there are minor issues
EXECUTION - Are the experiments and analyses performed with technical rigor to allow
confidence in the results?
No - there are minor issues
STATISTICS - Is the use of statistics in the manuscript appropriate?
No - there are issues with the statistics in the study
INTERPRETATION - Is the current interpretation/discussion of the results reasonable and not
overstated?
No - there are major issues
OVERALL MANUSCRIPT POTENTIAL - Is the current version of this work technically
sound? If not, can revisions be made to make the work technically sound?
Maybe - with major revisions
PEER REVIEWER COMMENTS:
GENERAL COMMENTS: The manuscript investigated an approach to predict hepatocellular
carcinoma (HCC) risk using artificial neural networks (ANNs) based on routine clinical and
demographic data from patients with chronic hepatitis B. It demonstrates a fine methodological
framework with comprehensive variable selection techniques and robust model evaluation
metrics. However, significant limitations impact its rigor and generalizability. The reliance on
data from a single center, combined with the absence of explicit training-test data splitting, raises
concerns about potential overfitting and bias. The small sample size further limits the model's
external validity. While the discussion effectively highlights the model's clinical relevance, it
underemphasizes critical limitations such as the lack of both internal and external validation
and geographic diversity. Overall, the study is a promising contribution to HCC prediction but
requires methodological refinement and broader validation to enhance its clinical applicability.
I provide detailed comments below for further improvement.
REQUESTED REVISIONS:
1. The manuscript briefly mentions the inclusion and exclusion criteria but does not specify
whether there were missing values in the final dataset or how missing or incomplete data were
handled. Additionally, no flowchart detailing participant recruitment is provided, which would
improve clarity and transparency.
2. While the manuscript claims no collinearity among variables, it lacks explicit details about
the diagnostics used to confirm this (e.g., variance inflation factors).
3. The study constructs five ANN models using different sets of predictors selected by various
variable selection methods. However, the authors do not compare the ANN models with simpler
approaches such as logistic regression, random forest (RF), support vector machines (SVM),
regularized regression, or best subset regression. Given the small sample size and limited number
of predictors, simpler models might perform comparably to the more complex ANN models.
4. The manuscript describes the construction of ANN models and their evaluation using
metrics such as AUC, calibration curves, and the Hosmer–Lemeshow test. However, it does not
explicitly describe whether the data was split into training and testing datasets. Training-test
splitting is essential to prevent overfitting and provide reliable performance metrics. Without
clear evidence of this step, the risk of overfitting is particularly high, especially for ANN models,
which are sensitive to noise in the data.
5. The discussion suggests that Model 5 is ready for clinical application without addressing the
inherent limitations of ANN models, such as potential overfitting and dependency on specific
sample characteristics.
6. Claims about the superiority of the ANN approach over traditional methods are not
substantiated by comparative analyses in the current study.
7. I recommend that the authors thoroughly revise the manuscript following the TRIPOD
guidelines for AI-based predictive models and include a completed TRIPOD checklist upon
resubmission.
Please review this paper.
user
Title: Development and validation of an interpretable machine learning model for predicting
the risk of hepatocellular carcinoma in patients with chronic hepatitis B: a case-control study
Version: 1 Date: 22 Nov 2024
Reviewer's report:
PEER REVIEWER ASSESSMENTS:
OBJECTIVE - Full research articles: is there a clear objective that addresses a testable research
question(s) (brief or other article types: is there a clear objective)?
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols) appropriate for
the objective?
No - there are minor issues
EXECUTION - Are the experiments and analyses performed with technical rigor to allow
confidence in the results?
No - there are minor issues
STATISTICS - Is the use of statistics in the manuscript appropriate?
No - there are issues with the statistics in the study
INTERPRETATION - Is the current interpretation/discussion of the results reasonable and not
overstated?
No - there are major issues
OVERALL MANUSCRIPT POTENTIAL - Is the current version of this work technically
sound? If not, can revisions be made to make the work technically sound?
Maybe - with major revisions
PEER REVIEWER COMMENTS:
GENERAL COMMENTS: The manuscript investigated an approach to predict hepatocellular
carcinoma (HCC) risk using artificial neural networks (ANNs) based on routine clinical and
demographic data from patients with chronic hepatitis B. It demonstrates a fine methodological
framework with comprehensive variable selection techniques and robust model evaluation
metrics. However, significant limitations impact its rigor and generalizability. The reliance on
data from a single center, combined with the absence of explicit training-test data splitting, raises
concerns about potential overfitting and bias. The small sample size further limits the model's
external validity. While the discussion effectively highlights the model's clinical relevance, it
underemphasizes critical limitations such as the lack of both internal and external validation
and geographic diversity. Overall, the study is a promising contribution to HCC prediction but
requires methodological refinement and broader validation to enhance its clinical applicability.
I provide detailed comments below for further improvement.
REQUESTED REVISIONS:
1. The manuscript briefly mentions the inclusion and exclusion criteria but does not specify
whether there were missing values in the final dataset or how missing or incomplete data were
handled. Additionally, no flowchart detailing participant recruitment is provided, which would
improve clarity and transparency.
2. While the manuscript claims no collinearity among variables, it lacks explicit details about
the diagnostics used to confirm this (e.g., variance inflation factors).
3. The study constructs five ANN models using different sets of predictors selected by various
variable selection methods. However, the authors do not compare the ANN models with simpler
approaches such as logistic regression, random forest (RF), support vector machines (SVM),
regularized regression, or best subset regression. Given the small sample size and limited number
of predictors, simpler models might perform comparably to the more complex ANN models.
4. The manuscript describes the construction of ANN models and their evaluation using
metrics such as AUC, calibration curves, and the Hosmer–Lemeshow test. However, it does not
explicitly describe whether the data was split into training and testing datasets. Training-test
splitting is essential to prevent overfitting and provide reliable performance metrics. Without
clear evidence of this step, the risk of overfitting is particularly high, especially for ANN models,
which are sensitive to noise in the data.
5. The discussion suggests that Model 5 is ready for clinical application without addressing the
inherent limitations of ANN models, such as potential overfitting and dependency on specific
sample characteristics.
6. Claims about the superiority of the ANN approach over traditional methods are not
substantiated by comparative analyses in the current study.
7. I recommend that the authors thoroughly revise the manuscript following the TRIPOD
guidelines for AI-based predictive models and include a completed TRIPOD checklist upon
resubmission.
assistant