Reviewer’s report
Title: 
Nomogram for predicting overall survival in colorectal cancer with distant
metastasis
Version: 
2 
Date: 
10 Feb 2021
Reviewer's report:
PEER REVIEWER ASSESSMENTS: 
OBJECTIVE - Full research articles: is there a clear objective that addresses a
testable research question(s) (brief or other article types: is there a clear
objective)?
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols)
appropriate for the objective?
Yes - the approach is appropriate
EXECUTION - Are the experiments and analyses performed with technical rigor
to allow confidence in the results?
Yes - experiments and analyses were performed appropriately
STATISTICS - Is the use of statistics in the manuscript appropriate?
Yes - appropriate statistical analyses have been used in the study
INTERPRETATION - Is the current interpretation/discussion of the results
reasonable and not overstated?
Yes - the author's interpretation is reasonable
OVERALL MANUSCRIPT POTENTIAL - Is the current version of this work
technically sound? If not, can revisions be made to make the work technically
sound?
Probably - with minor revisions
PEER REVIEWER COMMENTS: 
GENERAL COMMENTS: 
The paper is well written and uses the standard of care methods for developing
and creating a nomogram. ROC curves and area under the curve statistics were
evaluated. Calibration was examined. The results are limited by the fact that the
majority of the sample was excluded due to missing data (26,850 with missing
data, 14,197 with complete data). Hence, it is unknown how representative the
sample is of patients with colorectal cancer with distant metastasis. Many
variables may have been missing among people who did not receive much
treatment and hence did not have complete pathologic information. It might not
be unusual for those with advanced colorectal cancer to have limited therapy.
ADDITIONAL REQUESTS/SUGGESTIONS: 
Minor comments: Commas are misplaced in the flow diagram. 
The authors might have a stronger mention that the chemotherapy and radiation
information in SEER is incomplete and this is a limitation of using those
variables (https://seer.cancer.gov/data/ChemotherapyRadiation-SEER-
DUA.pdf).
It might be worth noting that those who did not receive surgery or chemotherapy
might not necessarily have been appropriate candidates for such therapy. Hence,
the surgical and chemotherapy estimates might not be causal. That is, if someone
is not an appropriate candidate for surgery or chemotherapy, receiving such
therapy might not improve their outcomes as the nomogram would suggest.
There is selection bias in the receipt of such therapy; SEER data might not
necessarily allow one to adjust for all variables associated with selection bias.
The authors might mention that there could be substantial missing data bias due
to the amount of missing data. The results might have been different had there
been more complete data.
Publons Reviewer Recognition
. Springer Nature can send verification of this
review directly to Publons (a subsidiary of Clarivate Analytics). If you would
like to take advantage of this service, please click on the “Yes” option below.
Your name, email address, title of the reviewed manuscript, name of the journal,
and date of your review submission (the “Review Data”) will then be transmitted
to Publons after the final decision on the manuscript has been made. If you have
already registered at Publons, they will notify you of the receipt of this review
and update your profile 
as per your settings and their policy. If you are not
registered with Publons, you will receive an email from them asking you to
register in order for them to be able to recognize your review on your new
profile page. Publons may use the Review Data to generate derivative metadata
for the benefit of Publons and you as a reviewer, carefully considering the
sensitivity of such information. For example, Publons may verify your record as
a reviewer by updating your profile published on its webservice if you have
registered for such service or help editors to identify candidate reviewers. Please
find the details of processing in Publons’ privacy policy
https://publons.com/about/terms
No
Declaration of competing interests
Please complete a declaration of competing interests, considering the following
questions:
1
. 
Have you in the past five years received reimbursements, fees, funding, or
salary from an organisation that may in any way gain or lose financially
from the publication of this manuscript, either now or in the future?
2
. 
Do you hold any stocks or shares in an organisation that may in any way
gain or lose financially from the publication of this manuscript, either now
or in the future?
3
. 
Do you hold or are you currently applying for any patents relating to the
content of the manuscript?
4
. 
Have you received reimbursements, fees, funding, or salary from an
organization that holds or has applied for patents relating to the content of
the manuscript?
5
. 
Do you have any other financial competing interests?
6
. 
Do you have any non-financial competing interests in relation to this
paper?
If you can answer no to all of the above, write 'I declare that I have no competing
interests' below. If your reply is yes to any, please give details below.
This reviewer has been recruited by a partner organization, Research Square.
Reviewers with declared or apparent competing interests are not utilized for
these reviews. This reviewer has agreed to publication of their comments online
under a Creative Commons Attribution License attributed to Research Square
and was paid a small honorarium for completing the review within a specified
timeframe. Honoraria for reviews such as this are paid regardless of the reviewer
recommendation.
Reviewer Publication Consent.
 I agree for my report to be made available
under an Open Access Creative Commons CC-BY License (
http://creativecommons.org/licenses/by/4.0
) if this manuscript is accepted for
publication. Any comments that I do not wish to be included in the published
report have been included as confidential comments to the editor, which will not
be published.
If you are not happy for us to publish this report, please contact the editorial
office before completing the review.
If you wish, you can include your name in your published report. Please note
you must decide whether to include your name at the start of the process and
confirm this decision whenever prompted. If you change your decision later, we
will not be able to publish your name.
I agree to the terms of the CC-BY 4.0 license; please 
do not
 publish my name
with my report. (default)
Is the study design appropriate to answer the research question (including the use
of appropriate controls), and are the conclusions supported by the evidence
presented?
Yes
Are the methods sufficiently described to allow the study to be repeated?
Yes
Is the use of statistics and treatment of uncertainties appropriate?
Yes
Is the presentation of the work clear?
No
Are the images in this manuscript (including electrophoretic gels and blots) free
from apparent manipulation?
Yes
Reviewer’s report
Title: 
Nomogram for predicting overall survival in colorectal cancer with distant
metastasis
Version: 
2 
Date: 
10 Feb 2021
Reviewer's report:
PEER REVIEWER ASSESSMENTS: 
OBJECTIVE - Full research articles: is there a clear objective that addresses a
testable research question(s) (brief or other article types: is there a clear
objective)?
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols)
appropriate for the objective?
Yes - the approach is appropriate
EXECUTION - Are the experiments and analyses performed with technical rigor
to allow confidence in the results?
Yes - experiments and analyses were performed appropriately
STATISTICS - Is the use of statistics in the manuscript appropriate?
Yes - appropriate statistical analyses have been used in the study
INTERPRETATION - Is the current interpretation/discussion of the results
reasonable and not overstated?
Yes - the author's interpretation is reasonable
OVERALL MANUSCRIPT POTENTIAL - Is the current version of this work
technically sound? If not, can revisions be made to make the work technically
sound?
Probably - with minor revisions
PEER REVIEWER COMMENTS: 
GENERAL COMMENTS: 
The paper is well written and uses the standard of care methods for developing
and creating a nomogram. ROC curves and area under the curve statistics were
evaluated. Calibration was examined. The results are limited by the fact that the
majority of the sample was excluded due to missing data (26,850 with missing
data, 14,197 with complete data). Hence, it is unknown how representative the
sample is of patients with colorectal cancer with distant metastasis. Many
variables may have been missing among people who did not receive much
treatment and hence did not have complete pathologic information. It might not
be unusual for those with advanced colorectal cancer to have limited therapy.
ADDITIONAL REQUESTS/SUGGESTIONS: 
Minor comments: Commas are misplaced in the flow diagram. 
The authors might have a stronger mention that the chemotherapy and radiation
information in SEER is incomplete and this is a limitation of using those
variables (https://seer.cancer.gov/data/ChemotherapyRadiation-SEER-
DUA.pdf).
It might be worth noting that those who did not receive surgery or chemotherapy
might not necessarily have been appropriate candidates for such therapy. Hence,
the surgical and chemotherapy estimates might not be causal. That is, if someone
is not an appropriate candidate for surgery or chemotherapy, receiving such
therapy might not improve their outcomes as the nomogram would suggest.
There is selection bias in the receipt of such therapy; SEER data might not
necessarily allow one to adjust for all variables associated with selection bias.
The authors might mention that there could be substantial missing data bias due
to the amount of missing data. The results might have been different had there
been more complete data.
Publons Reviewer Recognition
. Springer Nature can send verification of this
review directly to Publons (a subsidiary of Clarivate Analytics). If you would
like to take advantage of this service, please click on the “Yes” option below.
Your name, email address, title of the reviewed manuscript, name of the journal,
and date of your review submission (the “Review Data”) will then be transmitted
to Publons after the final decision on the manuscript has been made. If you have
already registered at Publons, they will notify you of the receipt of this review
and update your profile 
as per your settings and their policy. If you are not
registered with Publons, you will receive an email from them asking you to
register in order for them to be able to recognize your review on your new
profile page. Publons may use the Review Data to generate derivative metadata
for the benefit of Publons and you as a reviewer, carefully considering the
sensitivity of such information. For example, Publons may verify your record as
a reviewer by updating your profile published on its webservice if you have
registered for such service or help editors to identify candidate reviewers. Please
find the details of processing in Publons’ privacy policy
https://publons.com/about/terms
No
Declaration of competing interests
Please complete a declaration of competing interests, considering the following
questions:
1
. 
Have you in the past five years received reimbursements, fees, funding, or
salary from an organisation that may in any way gain or lose financially
from the publication of this manuscript, either now or in the future?
2
. 
Do you hold any stocks or shares in an organisation that may in any way
gain or lose financially from the publication of this manuscript, either now
or in the future?
3
. 
Do you hold or are you currently applying for any patents relating to the
content of the manuscript?
4
. 
Have you received reimbursements, fees, funding, or salary from an
organization that holds or has applied for patents relating to the content of
the manuscript?
5
. 
Do you have any other financial competing interests?
6
. 
Do you have any non-financial competing interests in relation to this
paper?
If you can answer no to all of the above, write 'I declare that I have no competing
interests' below. If your reply is yes to any, please give details below.
This reviewer has been recruited by a partner organization, Research Square.
Reviewers with declared or apparent competing interests are not utilized for
these reviews. This reviewer has agreed to publication of their comments online
under a Creative Commons Attribution License attributed to Research Square
and was paid a small honorarium for completing the review within a specified
timeframe. Honoraria for reviews such as this are paid regardless of the reviewer
recommendation.
Reviewer Publication Consent.
 I agree for my report to be made available
under an Open Access Creative Commons CC-BY License (
http://creativecommons.org/licenses/by/4.0
) if this manuscript is accepted for
publication. Any comments that I do not wish to be included in the published
report have been included as confidential comments to the editor, which will not
be published.
If you are not happy for us to publish this report, please contact the editorial
office before completing the review.
If you wish, you can include your name in your published report. Please note
you must decide whether to include your name at the start of the process and
confirm this decision whenever prompted. If you change your decision later, we
will not be able to publish your name.
I agree to the terms of the CC-BY 4.0 license; please 
do not
 publish my name
with my report. (default)
Is the study design appropriate to answer the research question (including the use
of appropriate controls), and are the conclusions supported by the evidence
presented?
Yes
Are the methods sufficiently described to allow the study to be repeated?
Yes
Is the use of statistics and treatment of uncertainties appropriate?
Yes
Is the presentation of the work clear?
No
Are the images in this manuscript (including electrophoretic gels and blots) free
from apparent manipulation?
Yes
Please review this paper.
user
Reviewer’s report
Title: 
Nomogram for predicting overall survival in colorectal cancer with distant
metastasis
Version: 
2 
Date: 
10 Feb 2021
Reviewer's report:
PEER REVIEWER ASSESSMENTS: 
OBJECTIVE - Full research articles: is there a clear objective that addresses a
testable research question(s) (brief or other article types: is there a clear
objective)?
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols)
appropriate for the objective?
Yes - the approach is appropriate
EXECUTION - Are the experiments and analyses performed with technical rigor
to allow confidence in the results?
Yes - experiments and analyses were performed appropriately
STATISTICS - Is the use of statistics in the manuscript appropriate?
Yes - appropriate statistical analyses have been used in the study
INTERPRETATION - Is the current interpretation/discussion of the results
reasonable and not overstated?
Yes - the author's interpretation is reasonable
OVERALL MANUSCRIPT POTENTIAL - Is the current version of this work
technically sound? If not, can revisions be made to make the work technically
sound?
Probably - with minor revisions
PEER REVIEWER COMMENTS: 
GENERAL COMMENTS: 
The paper is well written and uses the standard of care methods for developing
and creating a nomogram. ROC curves and area under the curve statistics were
evaluated. Calibration was examined. The results are limited by the fact that the
majority of the sample was excluded due to missing data (26,850 with missing
data, 14,197 with complete data). Hence, it is unknown how representative the
sample is of patients with colorectal cancer with distant metastasis. Many
variables may have been missing among people who did not receive much
treatment and hence did not have complete pathologic information. It might not
be unusual for those with advanced colorectal cancer to have limited therapy.
ADDITIONAL REQUESTS/SUGGESTIONS: 
Minor comments: Commas are misplaced in the flow diagram. 
The authors might have a stronger mention that the chemotherapy and radiation
information in SEER is incomplete and this is a limitation of using those
variables (https://seer.cancer.gov/data/ChemotherapyRadiation-SEER-
DUA.pdf).
It might be worth noting that those who did not receive surgery or chemotherapy
might not necessarily have been appropriate candidates for such therapy. Hence,
the surgical and chemotherapy estimates might not be causal. That is, if someone
is not an appropriate candidate for surgery or chemotherapy, receiving such
therapy might not improve their outcomes as the nomogram would suggest.
There is selection bias in the receipt of such therapy; SEER data might not
necessarily allow one to adjust for all variables associated with selection bias.
The authors might mention that there could be substantial missing data bias due
to the amount of missing data. The results might have been different had there
been more complete data.
Publons Reviewer Recognition
. Springer Nature can send verification of this
review directly to Publons (a subsidiary of Clarivate Analytics). If you would
like to take advantage of this service, please click on the “Yes” option below.
Your name, email address, title of the reviewed manuscript, name of the journal,
and date of your review submission (the “Review Data”) will then be transmitted
to Publons after the final decision on the manuscript has been made. If you have
already registered at Publons, they will notify you of the receipt of this review
and update your profile 
as per your settings and their policy. If you are not
registered with Publons, you will receive an email from them asking you to
register in order for them to be able to recognize your review on your new
profile page. Publons may use the Review Data to generate derivative metadata
for the benefit of Publons and you as a reviewer, carefully considering the
sensitivity of such information. For example, Publons may verify your record as
a reviewer by updating your profile published on its webservice if you have
registered for such service or help editors to identify candidate reviewers. Please
find the details of processing in Publons’ privacy policy
https://publons.com/about/terms
No
Declaration of competing interests
Please complete a declaration of competing interests, considering the following
questions:
1
. 
Have you in the past five years received reimbursements, fees, funding, or
salary from an organisation that may in any way gain or lose financially
from the publication of this manuscript, either now or in the future?
2
. 
Do you hold any stocks or shares in an organisation that may in any way
gain or lose financially from the publication of this manuscript, either now
or in the future?
3
. 
Do you hold or are you currently applying for any patents relating to the
content of the manuscript?
4
. 
Have you received reimbursements, fees, funding, or salary from an
organization that holds or has applied for patents relating to the content of
the manuscript?
5
. 
Do you have any other financial competing interests?
6
. 
Do you have any non-financial competing interests in relation to this
paper?
If you can answer no to all of the above, write 'I declare that I have no competing
interests' below. If your reply is yes to any, please give details below.
This reviewer has been recruited by a partner organization, Research Square.
Reviewers with declared or apparent competing interests are not utilized for
these reviews. This reviewer has agreed to publication of their comments online
under a Creative Commons Attribution License attributed to Research Square
and was paid a small honorarium for completing the review within a specified
timeframe. Honoraria for reviews such as this are paid regardless of the reviewer
recommendation.
Reviewer Publication Consent.
 I agree for my report to be made available
under an Open Access Creative Commons CC-BY License (
http://creativecommons.org/licenses/by/4.0
) if this manuscript is accepted for
publication. Any comments that I do not wish to be included in the published
report have been included as confidential comments to the editor, which will not
be published.
If you are not happy for us to publish this report, please contact the editorial
office before completing the review.
If you wish, you can include your name in your published report. Please note
you must decide whether to include your name at the start of the process and
confirm this decision whenever prompted. If you change your decision later, we
will not be able to publish your name.
I agree to the terms of the CC-BY 4.0 license; please 
do not
 publish my name
with my report. (default)
Is the study design appropriate to answer the research question (including the use
of appropriate controls), and are the conclusions supported by the evidence
presented?
Yes
Are the methods sufficiently described to allow the study to be repeated?
Yes
Is the use of statistics and treatment of uncertainties appropriate?
Yes
Is the presentation of the work clear?
No
Are the images in this manuscript (including electrophoretic gels and blots) free
from apparent manipulation?
Yes
assistant