Title: Performance and comparison of artificial intelligence and human experts in the detection
and classification of colonic polyps
Version: 1 Date: 19 May 2022
Reviewer's report:
General remarks
In this revised version of the manuscript the authors addressed most reviewer questions
satisfactorily. They did a good job with reporting the exclusion and inclusion criteria of
the different studies they analyzed in Table 2 and with reporting machine learning model
performance found in these studies in Table 3. When carefully reading the manuscript, the reader
is now able to extract key information about the different studies from these two tables.
However, some points have not been addressed and need to be clarified.
The major limitation of this manuscript, the fact that only 1 out of 16 studies has external
validation, remains. While the limitation is now mentioned by the authors upon request, the
claims made by the authors have not been adjusted. In particular, the conclusion “AI […] is
expected to improve the performance of young doctors” is not backed up by results.
In summary, the authors did a good job in summarizing 16 machine learning studies, but the
conclusion they draw at the end, which is that AI is expected to improve radiologists, is not
backed up by the results.
Remaining issues
Reporting AUC numbers without additional context has no value. Again, AUC is an abbreviation
for area under the curve. Please report which curve you are talking about. The authors failed to
address this issue in the revision.
One out of 16 studies (1/16 (6 %)) are not “a few” studies, it is just “one”. Using the term “a few”
is misleading. Also, the number of studies which include an external validation cohort is not a
conclusion, it should be reported in the results. The impact which the lack of external validation
has on the conclusions should be discussed in the discussion and mentioned in the conclusion.
Who expects AI to improve performance of radiologists? In the abstract the authors make it
sound like it this is the general opinion of the scientific community. However, in the discussion
the statement is much weaker and phrased as “we speculate” that AI improves the performance
of radiologists. This is misleading the audience and the authors have failed to address this issue
in the revision.
Title: Performance and comparison of artificial intelligence and human experts in the detection
and classification of colonic polyps
Version: 1 Date: 19 May 2022
Reviewer's report:
General remarks
In this revised version of the manuscript the authors addressed most reviewer questions
satisfactorily. They did a good job with reporting the exclusion and inclusion criteria of
the different studies they analyzed in Table 2 and with reporting machine learning model
performance found in these studies in Table 3. When carefully reading the manuscript, the reader
is now able to extract key information about the different studies from these two tables.
However, some points have not been addressed and need to be clarified.
The major limitation of this manuscript, the fact that only 1 out of 16 studies has external
validation, remains. While the limitation is now mentioned by the authors upon request, the
claims made by the authors have not been adjusted. In particular, the conclusion “AI […] is
expected to improve the performance of young doctors” is not backed up by results.
In summary, the authors did a good job in summarizing 16 machine learning studies, but the
conclusion they draw at the end, which is that AI is expected to improve radiologists, is not
backed up by the results.
Remaining issues
Reporting AUC numbers without additional context has no value. Again, AUC is an abbreviation
for area under the curve. Please report which curve you are talking about. The authors failed to
address this issue in the revision.
One out of 16 studies (1/16 (6 %)) are not “a few” studies, it is just “one”. Using the term “a few”
is misleading. Also, the number of studies which include an external validation cohort is not a
conclusion, it should be reported in the results. The impact which the lack of external validation
has on the conclusions should be discussed in the discussion and mentioned in the conclusion.
Who expects AI to improve performance of radiologists? In the abstract the authors make it
sound like it this is the general opinion of the scientific community. However, in the discussion
the statement is much weaker and phrased as “we speculate” that AI improves the performance
of radiologists. This is misleading the audience and the authors have failed to address this issue
in the revision.
Please review this paper.
user
Title: Performance and comparison of artificial intelligence and human experts in the detection
and classification of colonic polyps
Version: 1 Date: 19 May 2022
Reviewer's report:
General remarks
In this revised version of the manuscript the authors addressed most reviewer questions
satisfactorily. They did a good job with reporting the exclusion and inclusion criteria of
the different studies they analyzed in Table 2 and with reporting machine learning model
performance found in these studies in Table 3. When carefully reading the manuscript, the reader
is now able to extract key information about the different studies from these two tables.
However, some points have not been addressed and need to be clarified.
The major limitation of this manuscript, the fact that only 1 out of 16 studies has external
validation, remains. While the limitation is now mentioned by the authors upon request, the
claims made by the authors have not been adjusted. In particular, the conclusion “AI […] is
expected to improve the performance of young doctors” is not backed up by results.
In summary, the authors did a good job in summarizing 16 machine learning studies, but the
conclusion they draw at the end, which is that AI is expected to improve radiologists, is not
backed up by the results.
Remaining issues
Reporting AUC numbers without additional context has no value. Again, AUC is an abbreviation
for area under the curve. Please report which curve you are talking about. The authors failed to
address this issue in the revision.
One out of 16 studies (1/16 (6 %)) are not “a few” studies, it is just “one”. Using the term “a few”
is misleading. Also, the number of studies which include an external validation cohort is not a
conclusion, it should be reported in the results. The impact which the lack of external validation
has on the conclusions should be discussed in the discussion and mentioned in the conclusion.
Who expects AI to improve performance of radiologists? In the abstract the authors make it
sound like it this is the general opinion of the scientific community. However, in the discussion
the statement is much weaker and phrased as “we speculate” that AI improves the performance
of radiologists. This is misleading the audience and the authors have failed to address this issue
in the revision.
assistant