Title:  Development and clinical validation of deep learning-based Immunohistochemistry
prediction models for subtyping and staging of gastrointestinal cancers
Version: 4 Date: 04 Mar 2025
Reviewer's report:
In their manuscript, "Development and clinical validation of deep learning based virtual
Immunohistochemistry staining models in gastrointestinal cancers," the authors describe a
pipeline for training AI models to generate virtual IHC (vIHC)output from H&E whole slide
images (WSIs). The training dataset comprises 134 WSI images (H&E and IHC pairs). Five
models were trained to create vIHC for p40, pan-cytokeratin, desmin, P53, and Ki-67. An
additional dataset of 150 WSIs from 30 patients was used to test the model using pathologist
reads. Pathologists read each case using vIHC and then real IHC with a 2-week washout period.
The authors report high consistency for desmin, panCK, and p40 and moderate for p53.
In this manuscript, the methods are poorly described, and much methodology is simply absent or
poorly described. The metrics for evaluating the performance of the models would not capture
the quality of the vIHC or are poorly described.
MAJOR:
=====
1. RESULTS / TABLE S3: This provides some information about the testing dataset, but is
short on details. It is impossible to understand how challenging this task was without knowing
more about the nature of those cases. The difference between, for example, predicting p40 in
well-differentiated squamous cell carcinoma (a p40 would NEVER be needed) v. a poorly-
differentiated carcinoma (where a p40 might be vital) is vast. The former is a pointless exercise,
and the latter is almost the only clinically relevant application for p40 in the GI tract (outside of
establishing a diagnosis of adenosquamous). Without knowing the composition of the datasets,
it is impossible to evaluate any of the results in the manuscript or establish their impact.
2. METHODS: The authors state they "[aim] to create an automatic pipeline," but it would
appear that they are using HEMnet [reference 20] which was created by a different group.
Is HEMnet the only model? Was it just used for registration? Little detail is provided about
how they implemented their pipeline or models in the methods. No details are provided about
how the model works or how it was trained. Simply referencing another paper for the model
does not provide sufficient detail about the training of these models. While this is collectively
a MAJOR issue with the manuscript, additional (but not exhaustive) specific concerns with
unclear methods are also included as MINOR issues.
3. METHODS: "three pathologists... were involved into a real-world clinical scenario to write
pathology reports according to AI-IHC and Real-IHC, respectively." T staging of cancer cases
is, in almost all cases, performed on H&E slides are would RARELY benefit from the addition of
p40, pan-cytokeratin, and desmin. Desmin might be useful, but I have "needed" it for T staging
about twice in 12 years, i.e. rare utility. Overall, T-stage is a poor metric for establishing clinical
utility for the models.
4. METHODS: How exactly was the Ki-67 PI calculated in the study? Was the pathologist
estimating it "by eye" from the IHC and vIHC? Was it manually counted or calculated from the
IHC and vIHC using digital image analysis? The methods provided are not clear.
MINOR:
=====
1. INTRODUCTION: "The fifth edition of the WHO recommends routinely investigating these
five fundamental biomarkers in GI cancer for pathology clinical practice: Pan-CK, P40, Desmin,
P53, Ki-67, which are essential for carcinoma subtype diagnosis and treatment of cancer." To
say that these stains are "routinely investigated" is to misunderstand the use of these IHC in
GI pathology. These are stains that are used in very specific circumstances for very specific
reasons, or in the case of Ki-67, for NETs. None of them have a broad diagnostic ("subtyping"),
prognostic, or predictive role in GI cancer to the extent the authors portray.
2. MATERIALS AND METHODS: Whole-slide images preparation: "Our study focused on
five essential biomarkers in pathology clinical practice, namely P40 (scanned by KFBIO), Pan-
CK (scanned by 3DHISTECH), Desmin (scanned by 3DHISTECH), P53 (scanned by KFBIO),
and Ki-67 (scanned by KFBIO)." Several items:
a. Why were multiple scanner models used?
b. What file format and compression were used?
c. What were the microns per pixel (mpp) for the scans?
3. MATERIALS AND METHODS: Except for "VIA" and "SPSS" no other software is
specifically identified in the methods.
4. MATERIALS AND METHODS: Automatic annotation of target regions: "In order to
determine the range of IHC slice sequences that can be matched precisely."  The method for
determining how closely the slices match is not provided in the methods beyond "[calculating]
the mutual information of different adjacent slices."
5. MATERIALS AND METHODS: Double check by pathologists: "Pre-cropped negative and
positive tiles were reviewed by pathologists to correct minor errors."  What are negative and
positive tiles? How are the pathologists "correcting" minor errors?
6. MATERIALS AND METHODS: Double check by pathologists: What is the VIA software
(no reference provided) and what did the pathologists use it to do exactly?
7. MATERIALS AND METHODS: Model development and test: "The models based on semi-
supervised deep learning were trained to predict positive tiles using the cropped image tiles from
the corrected VIA labels."  Is this HEMnet of some other model? Also, positivity at a tile level
is extremely crude.
8. MATERIALS AND METHODS: Model development and test: How was the accuracy of the
models tested? This is the major readout of the study, yet no details for the metric are provided.
9. RESULTS: "During this process, the time of pathology reviewing ranges from 14±8
minutes...." Per WSI?
10. RESULTS: "As for the prediction and quantification of Ki-67, the proliferation index reports
according to AI-IHC presented a difference ranging from 17.35%±16.2%." This is a huge range?
Why not analyze with kappa or ICC? This looks like very poor performance for Ki-67.
11. FIGURE 4: A low-power image tells the reader nothing about the performance of vIHC.
High-power images are required to determine whether the models are good.
12. TABLE S3: A null phenotype for p53 is also aberrant and not uncommon. Did none of these
cases exhibit a null phenotype?
Title:  Development and clinical validation of deep learning-based Immunohistochemistry
prediction models for subtyping and staging of gastrointestinal cancers
Version: 4 Date: 04 Mar 2025
Reviewer's report:
In their manuscript, "Development and clinical validation of deep learning based virtual
Immunohistochemistry staining models in gastrointestinal cancers," the authors describe a
pipeline for training AI models to generate virtual IHC (vIHC)output from H&E whole slide
images (WSIs). The training dataset comprises 134 WSI images (H&E and IHC pairs). Five
models were trained to create vIHC for p40, pan-cytokeratin, desmin, P53, and Ki-67. An
additional dataset of 150 WSIs from 30 patients was used to test the model using pathologist
reads. Pathologists read each case using vIHC and then real IHC with a 2-week washout period.
The authors report high consistency for desmin, panCK, and p40 and moderate for p53.
In this manuscript, the methods are poorly described, and much methodology is simply absent or
poorly described. The metrics for evaluating the performance of the models would not capture
the quality of the vIHC or are poorly described.
MAJOR:
=====
1. RESULTS / TABLE S3: This provides some information about the testing dataset, but is
short on details. It is impossible to understand how challenging this task was without knowing
more about the nature of those cases. The difference between, for example, predicting p40 in
well-differentiated squamous cell carcinoma (a p40 would NEVER be needed) v. a poorly-
differentiated carcinoma (where a p40 might be vital) is vast. The former is a pointless exercise,
and the latter is almost the only clinically relevant application for p40 in the GI tract (outside of
establishing a diagnosis of adenosquamous). Without knowing the composition of the datasets,
it is impossible to evaluate any of the results in the manuscript or establish their impact.
2. METHODS: The authors state they "[aim] to create an automatic pipeline," but it would
appear that they are using HEMnet [reference 20] which was created by a different group.
Is HEMnet the only model? Was it just used for registration? Little detail is provided about
how they implemented their pipeline or models in the methods. No details are provided about
how the model works or how it was trained. Simply referencing another paper for the model
does not provide sufficient detail about the training of these models. While this is collectively
a MAJOR issue with the manuscript, additional (but not exhaustive) specific concerns with
unclear methods are also included as MINOR issues.
3. METHODS: "three pathologists... were involved into a real-world clinical scenario to write
pathology reports according to AI-IHC and Real-IHC, respectively." T staging of cancer cases
is, in almost all cases, performed on H&E slides are would RARELY benefit from the addition of
p40, pan-cytokeratin, and desmin. Desmin might be useful, but I have "needed" it for T staging
about twice in 12 years, i.e. rare utility. Overall, T-stage is a poor metric for establishing clinical
utility for the models.
4. METHODS: How exactly was the Ki-67 PI calculated in the study? Was the pathologist
estimating it "by eye" from the IHC and vIHC? Was it manually counted or calculated from the
IHC and vIHC using digital image analysis? The methods provided are not clear.
MINOR:
=====
1. INTRODUCTION: "The fifth edition of the WHO recommends routinely investigating these
five fundamental biomarkers in GI cancer for pathology clinical practice: Pan-CK, P40, Desmin,
P53, Ki-67, which are essential for carcinoma subtype diagnosis and treatment of cancer." To
say that these stains are "routinely investigated" is to misunderstand the use of these IHC in
GI pathology. These are stains that are used in very specific circumstances for very specific
reasons, or in the case of Ki-67, for NETs. None of them have a broad diagnostic ("subtyping"),
prognostic, or predictive role in GI cancer to the extent the authors portray.
2. MATERIALS AND METHODS: Whole-slide images preparation: "Our study focused on
five essential biomarkers in pathology clinical practice, namely P40 (scanned by KFBIO), Pan-
CK (scanned by 3DHISTECH), Desmin (scanned by 3DHISTECH), P53 (scanned by KFBIO),
and Ki-67 (scanned by KFBIO)." Several items:
a. Why were multiple scanner models used?
b. What file format and compression were used?
c. What were the microns per pixel (mpp) for the scans?
3. MATERIALS AND METHODS: Except for "VIA" and "SPSS" no other software is
specifically identified in the methods.
4. MATERIALS AND METHODS: Automatic annotation of target regions: "In order to
determine the range of IHC slice sequences that can be matched precisely."  The method for
determining how closely the slices match is not provided in the methods beyond "[calculating]
the mutual information of different adjacent slices."
5. MATERIALS AND METHODS: Double check by pathologists: "Pre-cropped negative and
positive tiles were reviewed by pathologists to correct minor errors."  What are negative and
positive tiles? How are the pathologists "correcting" minor errors?
6. MATERIALS AND METHODS: Double check by pathologists: What is the VIA software
(no reference provided) and what did the pathologists use it to do exactly?
7. MATERIALS AND METHODS: Model development and test: "The models based on semi-
supervised deep learning were trained to predict positive tiles using the cropped image tiles from
the corrected VIA labels."  Is this HEMnet of some other model? Also, positivity at a tile level
is extremely crude.
8. MATERIALS AND METHODS: Model development and test: How was the accuracy of the
models tested? This is the major readout of the study, yet no details for the metric are provided.
9. RESULTS: "During this process, the time of pathology reviewing ranges from 14±8
minutes...." Per WSI?
10. RESULTS: "As for the prediction and quantification of Ki-67, the proliferation index reports
according to AI-IHC presented a difference ranging from 17.35%±16.2%." This is a huge range?
Why not analyze with kappa or ICC? This looks like very poor performance for Ki-67.
11. FIGURE 4: A low-power image tells the reader nothing about the performance of vIHC.
High-power images are required to determine whether the models are good.
12. TABLE S3: A null phenotype for p53 is also aberrant and not uncommon. Did none of these
cases exhibit a null phenotype?
Please review this paper.
user
Title:  Development and clinical validation of deep learning-based Immunohistochemistry
prediction models for subtyping and staging of gastrointestinal cancers
Version: 4 Date: 04 Mar 2025
Reviewer's report:
In their manuscript, "Development and clinical validation of deep learning based virtual
Immunohistochemistry staining models in gastrointestinal cancers," the authors describe a
pipeline for training AI models to generate virtual IHC (vIHC)output from H&E whole slide
images (WSIs). The training dataset comprises 134 WSI images (H&E and IHC pairs). Five
models were trained to create vIHC for p40, pan-cytokeratin, desmin, P53, and Ki-67. An
additional dataset of 150 WSIs from 30 patients was used to test the model using pathologist
reads. Pathologists read each case using vIHC and then real IHC with a 2-week washout period.
The authors report high consistency for desmin, panCK, and p40 and moderate for p53.
In this manuscript, the methods are poorly described, and much methodology is simply absent or
poorly described. The metrics for evaluating the performance of the models would not capture
the quality of the vIHC or are poorly described.
MAJOR:
=====
1. RESULTS / TABLE S3: This provides some information about the testing dataset, but is
short on details. It is impossible to understand how challenging this task was without knowing
more about the nature of those cases. The difference between, for example, predicting p40 in
well-differentiated squamous cell carcinoma (a p40 would NEVER be needed) v. a poorly-
differentiated carcinoma (where a p40 might be vital) is vast. The former is a pointless exercise,
and the latter is almost the only clinically relevant application for p40 in the GI tract (outside of
establishing a diagnosis of adenosquamous). Without knowing the composition of the datasets,
it is impossible to evaluate any of the results in the manuscript or establish their impact.
2. METHODS: The authors state they "[aim] to create an automatic pipeline," but it would
appear that they are using HEMnet [reference 20] which was created by a different group.
Is HEMnet the only model? Was it just used for registration? Little detail is provided about
how they implemented their pipeline or models in the methods. No details are provided about
how the model works or how it was trained. Simply referencing another paper for the model
does not provide sufficient detail about the training of these models. While this is collectively
a MAJOR issue with the manuscript, additional (but not exhaustive) specific concerns with
unclear methods are also included as MINOR issues.
3. METHODS: "three pathologists... were involved into a real-world clinical scenario to write
pathology reports according to AI-IHC and Real-IHC, respectively." T staging of cancer cases
is, in almost all cases, performed on H&E slides are would RARELY benefit from the addition of
p40, pan-cytokeratin, and desmin. Desmin might be useful, but I have "needed" it for T staging
about twice in 12 years, i.e. rare utility. Overall, T-stage is a poor metric for establishing clinical
utility for the models.
4. METHODS: How exactly was the Ki-67 PI calculated in the study? Was the pathologist
estimating it "by eye" from the IHC and vIHC? Was it manually counted or calculated from the
IHC and vIHC using digital image analysis? The methods provided are not clear.
MINOR:
=====
1. INTRODUCTION: "The fifth edition of the WHO recommends routinely investigating these
five fundamental biomarkers in GI cancer for pathology clinical practice: Pan-CK, P40, Desmin,
P53, Ki-67, which are essential for carcinoma subtype diagnosis and treatment of cancer." To
say that these stains are "routinely investigated" is to misunderstand the use of these IHC in
GI pathology. These are stains that are used in very specific circumstances for very specific
reasons, or in the case of Ki-67, for NETs. None of them have a broad diagnostic ("subtyping"),
prognostic, or predictive role in GI cancer to the extent the authors portray.
2. MATERIALS AND METHODS: Whole-slide images preparation: "Our study focused on
five essential biomarkers in pathology clinical practice, namely P40 (scanned by KFBIO), Pan-
CK (scanned by 3DHISTECH), Desmin (scanned by 3DHISTECH), P53 (scanned by KFBIO),
and Ki-67 (scanned by KFBIO)." Several items:
a. Why were multiple scanner models used?
b. What file format and compression were used?
c. What were the microns per pixel (mpp) for the scans?
3. MATERIALS AND METHODS: Except for "VIA" and "SPSS" no other software is
specifically identified in the methods.
4. MATERIALS AND METHODS: Automatic annotation of target regions: "In order to
determine the range of IHC slice sequences that can be matched precisely."  The method for
determining how closely the slices match is not provided in the methods beyond "[calculating]
the mutual information of different adjacent slices."
5. MATERIALS AND METHODS: Double check by pathologists: "Pre-cropped negative and
positive tiles were reviewed by pathologists to correct minor errors."  What are negative and
positive tiles? How are the pathologists "correcting" minor errors?
6. MATERIALS AND METHODS: Double check by pathologists: What is the VIA software
(no reference provided) and what did the pathologists use it to do exactly?
7. MATERIALS AND METHODS: Model development and test: "The models based on semi-
supervised deep learning were trained to predict positive tiles using the cropped image tiles from
the corrected VIA labels."  Is this HEMnet of some other model? Also, positivity at a tile level
is extremely crude.
8. MATERIALS AND METHODS: Model development and test: How was the accuracy of the
models tested? This is the major readout of the study, yet no details for the metric are provided.
9. RESULTS: "During this process, the time of pathology reviewing ranges from 14±8
minutes...." Per WSI?
10. RESULTS: "As for the prediction and quantification of Ki-67, the proliferation index reports
according to AI-IHC presented a difference ranging from 17.35%±16.2%." This is a huge range?
Why not analyze with kappa or ICC? This looks like very poor performance for Ki-67.
11. FIGURE 4: A low-power image tells the reader nothing about the performance of vIHC.
High-power images are required to determine whether the models are good.
12. TABLE S3: A null phenotype for p53 is also aberrant and not uncommon. Did none of these
cases exhibit a null phenotype?
assistant