Title: A simplified multiplex methylated DNA testing for early detection of colorectal cancer
in stool DNA
Version: 0 Date: 21 Jul 2022
Reviewer:  Mai-britt Worm Orntoft
Reviewer's report:
Review of the manuscript entitled: A simplified multiplex methylated DNA testing for early
detection of colorectal cancer in stool DNA, by Dai, Y. el al
The subject of colorectal cancer screening, especially in developing countries where health care
is challenged by low resources, is very interesting.
The aim of the present manuscript is to describe the optimization of the qPCR based test for
early colorectal cancer (CRC) detection in stool samples and present data on it’s performance
on CRC patients and control individuals. As such it should preferably adhere to the published
REMARK criteria (https://www.nature.com/articles/ncponc0252/tables/1).
The test is called ColoDefense 2.0 and differs from ColoDefense 1.0 by especially one factor:
The qPCR reaction in ColoDefense 2.0 is a single reaction and not triplicates as is the common
standard and used in ColoDefense 1.0. Further the paper presents a small study on how much
DNA degrades over time, in the stool sample storage tubes, to evaluate how quickly stools
samples need to be evaluated in a clinical setting.
The study design is a retrospective study, where stool samples are collected just before bowel
preparation or colonoscopy, but analysed when the result from the colonoscopy is known.
The authors mention a training cohort of 142 CRC patients 14 AA patients and 85 controls
subjects. Further a validation cohort of 102 CRC patients, 20 AA patients, 64 SP and 102 control
individuals is mentioned.
Patients
For both groups inclusion and exclusion criteria and how they were recruited are omitted. Did
they come from a cohort of symptomatic patients referred for colonoscopy, or did they come
from a screening cohort? Could e.g. patients with earlier CRC be included or not? Further lacks
the description of whether control subjects had other findings in their colon than a clean colon.
1) Would authors please present these data on exclusion, inclusion etc. and on colonoscopy
results in control patients and whether there was any matching between cases and controls on
e.g. age.
2) Further, would authors present the mean age and gender in controls versus cases in both
cohorts?
In my opinion young, clean colon patients are probably a poor control group for at test aimed at
elderly CRC screening patients; therefore, it is reasonable to ask whether the control group here
is too young and healthy? Elderly individuals are prone to diverticular disease, inflammation,
infection etc. that can all induce epigenetic aberrations which give false positive test results.
Also, older individuals can present with epigenetic aberrations due to age alone. To reflect this,
the most suitable control group is probably elderly patients, with diverse colonoscopy findings,
as these will reflect the true specificity for a test aimed at an elderly cohort. If young, clean colon
individuals are used, this can ultimately lead to a falsely high specificity.
Analyses
The test cut-offs are different from the ColoDefense 2.0: If ACTB is above 43.0 the test is invalid
and if SEPT9 is below 38,0 or SCD2 is below 40,0 the test is positive. However, authors do
not describe how these exact cut-offs are determined. It would be standard to use the training
cohort to find the cut-offs based on results on cases and controls, and then test the cut-offs in
the validation cohort.
3) Would authors please describe how they decide where the qPCR cut-off should be set, and
how the training and validation cohort are used in this process (if this was the case?)?
4) From what authors describe, the only difference between the validation and training cohort are
that in the training cohort the ColoDefense 1,0 and 2.0 are tested on the same samples an render
similar performance. In my opinion, this is not at training cohort, but a comparison to an already
published test in a subset of patients. Thus, it is obsolete to describe the use of ColoDefense 2.0
in the remaining patients as a validation, unless other differences characterize the analyses in
this cohort or it included another type of patients?
If this is not the case, the two cohorts should be merged to one, and the side-by-side comparison
described on a subset of patients.
Regarding the analytical performance of the ColoDefense 2.0 assay, authors have tested LoD
by a dilution series of fully methylated DNA in a background of fully unmethylated DNA.
5) Would authors please supple how high the background concentration of unmethylated DNA
was in the samples? The higher the background concentration, the higher the risk of false positive
results, that affects the assay specificity.
6) Authors also need to present data on the Limit of Blanc (LoB). In screening testing, a
negative result can be as important as a positive result and the LoB is important to ensure a
high specificity.
7) It is customary to supply the qPCR assay data and location on the DNA strand for peers to
evaluate the assay performance in their lab. Why is this data not presented?
8) Was the cut-off for a positive/negative assay set in a blinded manner or was the analyst aware
of colonoscopy results in the training and validation cohort?
I find the test on DNA degradation in the collection tubes interesting and necessary to conclude
that stool samples can be collected peripherally and analysed centrally in a reasonable work-
flow.
Results
The Results are presented as text and figures.
9) However, authors do not present how many reactions were invalid due to technical errors or
to little DNA? How about quality control indicators? How much DNA was lost in the bilsufite
conversion? How many samples were lost in the process or inconclusive? What characterized
the false positive and false negative test result individuals?
10) I find it peculiar that there is no difference in test performance across CRC stages? Usually,
the sensitivity increases with stage. Please, would authors touch upon this in the discussion?
Authors argue that the use of a single qPCR reaction (ColoDefense 2.0) compared to triplicate
reactions (ColoDefense 1,.) in the subset of patients in the training cohort does not affect the test
performance and hence it is an acceptable alternative. This could be true but is controversial, as
most literature agrees that at least two technical replicates should be made in a qPCR reaction.
11) Would authors please reflect upon why no other published PCR-based assays for CRC
detection rely on only one technical replicate. If the performance and quality is as good as authors
state, then I would expect all lab’s to only perform qPCR as single reactions.
12) A limitations section is lacking in the discussion:
Fx authors mention the need for cheap screening tools in developing countries. Is this test
comparable to the standard iFOBT/FIT test in performance and price and hence a better
alternative?
The test has not yet been applied to an actual screening cohort in a prospective trial. Is there
any plans to do this? The real performance cannot be disclosed until the test has been tried in
a proper cohort of screening patients.
Title: A simplified multiplex methylated DNA testing for early detection of colorectal cancer
in stool DNA
Version: 0 Date: 21 Jul 2022
Reviewer:  Mai-britt Worm Orntoft
Reviewer's report:
Review of the manuscript entitled: A simplified multiplex methylated DNA testing for early
detection of colorectal cancer in stool DNA, by Dai, Y. el al
The subject of colorectal cancer screening, especially in developing countries where health care
is challenged by low resources, is very interesting.
The aim of the present manuscript is to describe the optimization of the qPCR based test for
early colorectal cancer (CRC) detection in stool samples and present data on it’s performance
on CRC patients and control individuals. As such it should preferably adhere to the published
REMARK criteria (https://www.nature.com/articles/ncponc0252/tables/1).
The test is called ColoDefense 2.0 and differs from ColoDefense 1.0 by especially one factor:
The qPCR reaction in ColoDefense 2.0 is a single reaction and not triplicates as is the common
standard and used in ColoDefense 1.0. Further the paper presents a small study on how much
DNA degrades over time, in the stool sample storage tubes, to evaluate how quickly stools
samples need to be evaluated in a clinical setting.
The study design is a retrospective study, where stool samples are collected just before bowel
preparation or colonoscopy, but analysed when the result from the colonoscopy is known.
The authors mention a training cohort of 142 CRC patients 14 AA patients and 85 controls
subjects. Further a validation cohort of 102 CRC patients, 20 AA patients, 64 SP and 102 control
individuals is mentioned.
Patients
For both groups inclusion and exclusion criteria and how they were recruited are omitted. Did
they come from a cohort of symptomatic patients referred for colonoscopy, or did they come
from a screening cohort? Could e.g. patients with earlier CRC be included or not? Further lacks
the description of whether control subjects had other findings in their colon than a clean colon.
1) Would authors please present these data on exclusion, inclusion etc. and on colonoscopy
results in control patients and whether there was any matching between cases and controls on
e.g. age.
2) Further, would authors present the mean age and gender in controls versus cases in both
cohorts?
In my opinion young, clean colon patients are probably a poor control group for at test aimed at
elderly CRC screening patients; therefore, it is reasonable to ask whether the control group here
is too young and healthy? Elderly individuals are prone to diverticular disease, inflammation,
infection etc. that can all induce epigenetic aberrations which give false positive test results.
Also, older individuals can present with epigenetic aberrations due to age alone. To reflect this,
the most suitable control group is probably elderly patients, with diverse colonoscopy findings,
as these will reflect the true specificity for a test aimed at an elderly cohort. If young, clean colon
individuals are used, this can ultimately lead to a falsely high specificity.
Analyses
The test cut-offs are different from the ColoDefense 2.0: If ACTB is above 43.0 the test is invalid
and if SEPT9 is below 38,0 or SCD2 is below 40,0 the test is positive. However, authors do
not describe how these exact cut-offs are determined. It would be standard to use the training
cohort to find the cut-offs based on results on cases and controls, and then test the cut-offs in
the validation cohort.
3) Would authors please describe how they decide where the qPCR cut-off should be set, and
how the training and validation cohort are used in this process (if this was the case?)?
4) From what authors describe, the only difference between the validation and training cohort are
that in the training cohort the ColoDefense 1,0 and 2.0 are tested on the same samples an render
similar performance. In my opinion, this is not at training cohort, but a comparison to an already
published test in a subset of patients. Thus, it is obsolete to describe the use of ColoDefense 2.0
in the remaining patients as a validation, unless other differences characterize the analyses in
this cohort or it included another type of patients?
If this is not the case, the two cohorts should be merged to one, and the side-by-side comparison
described on a subset of patients.
Regarding the analytical performance of the ColoDefense 2.0 assay, authors have tested LoD
by a dilution series of fully methylated DNA in a background of fully unmethylated DNA.
5) Would authors please supple how high the background concentration of unmethylated DNA
was in the samples? The higher the background concentration, the higher the risk of false positive
results, that affects the assay specificity.
6) Authors also need to present data on the Limit of Blanc (LoB). In screening testing, a
negative result can be as important as a positive result and the LoB is important to ensure a
high specificity.
7) It is customary to supply the qPCR assay data and location on the DNA strand for peers to
evaluate the assay performance in their lab. Why is this data not presented?
8) Was the cut-off for a positive/negative assay set in a blinded manner or was the analyst aware
of colonoscopy results in the training and validation cohort?
I find the test on DNA degradation in the collection tubes interesting and necessary to conclude
that stool samples can be collected peripherally and analysed centrally in a reasonable work-
flow.
Results
The Results are presented as text and figures.
9) However, authors do not present how many reactions were invalid due to technical errors or
to little DNA? How about quality control indicators? How much DNA was lost in the bilsufite
conversion? How many samples were lost in the process or inconclusive? What characterized
the false positive and false negative test result individuals?
10) I find it peculiar that there is no difference in test performance across CRC stages? Usually,
the sensitivity increases with stage. Please, would authors touch upon this in the discussion?
Authors argue that the use of a single qPCR reaction (ColoDefense 2.0) compared to triplicate
reactions (ColoDefense 1,.) in the subset of patients in the training cohort does not affect the test
performance and hence it is an acceptable alternative. This could be true but is controversial, as
most literature agrees that at least two technical replicates should be made in a qPCR reaction.
11) Would authors please reflect upon why no other published PCR-based assays for CRC
detection rely on only one technical replicate. If the performance and quality is as good as authors
state, then I would expect all lab’s to only perform qPCR as single reactions.
12) A limitations section is lacking in the discussion:
Fx authors mention the need for cheap screening tools in developing countries. Is this test
comparable to the standard iFOBT/FIT test in performance and price and hence a better
alternative?
The test has not yet been applied to an actual screening cohort in a prospective trial. Is there
any plans to do this? The real performance cannot be disclosed until the test has been tried in
a proper cohort of screening patients.
Please review this paper.
user
Title: A simplified multiplex methylated DNA testing for early detection of colorectal cancer
in stool DNA
Version: 0 Date: 21 Jul 2022
Reviewer:  Mai-britt Worm Orntoft
Reviewer's report:
Review of the manuscript entitled: A simplified multiplex methylated DNA testing for early
detection of colorectal cancer in stool DNA, by Dai, Y. el al
The subject of colorectal cancer screening, especially in developing countries where health care
is challenged by low resources, is very interesting.
The aim of the present manuscript is to describe the optimization of the qPCR based test for
early colorectal cancer (CRC) detection in stool samples and present data on it’s performance
on CRC patients and control individuals. As such it should preferably adhere to the published
REMARK criteria (https://www.nature.com/articles/ncponc0252/tables/1).
The test is called ColoDefense 2.0 and differs from ColoDefense 1.0 by especially one factor:
The qPCR reaction in ColoDefense 2.0 is a single reaction and not triplicates as is the common
standard and used in ColoDefense 1.0. Further the paper presents a small study on how much
DNA degrades over time, in the stool sample storage tubes, to evaluate how quickly stools
samples need to be evaluated in a clinical setting.
The study design is a retrospective study, where stool samples are collected just before bowel
preparation or colonoscopy, but analysed when the result from the colonoscopy is known.
The authors mention a training cohort of 142 CRC patients 14 AA patients and 85 controls
subjects. Further a validation cohort of 102 CRC patients, 20 AA patients, 64 SP and 102 control
individuals is mentioned.
Patients
For both groups inclusion and exclusion criteria and how they were recruited are omitted. Did
they come from a cohort of symptomatic patients referred for colonoscopy, or did they come
from a screening cohort? Could e.g. patients with earlier CRC be included or not? Further lacks
the description of whether control subjects had other findings in their colon than a clean colon.
1) Would authors please present these data on exclusion, inclusion etc. and on colonoscopy
results in control patients and whether there was any matching between cases and controls on
e.g. age.
2) Further, would authors present the mean age and gender in controls versus cases in both
cohorts?
In my opinion young, clean colon patients are probably a poor control group for at test aimed at
elderly CRC screening patients; therefore, it is reasonable to ask whether the control group here
is too young and healthy? Elderly individuals are prone to diverticular disease, inflammation,
infection etc. that can all induce epigenetic aberrations which give false positive test results.
Also, older individuals can present with epigenetic aberrations due to age alone. To reflect this,
the most suitable control group is probably elderly patients, with diverse colonoscopy findings,
as these will reflect the true specificity for a test aimed at an elderly cohort. If young, clean colon
individuals are used, this can ultimately lead to a falsely high specificity.
Analyses
The test cut-offs are different from the ColoDefense 2.0: If ACTB is above 43.0 the test is invalid
and if SEPT9 is below 38,0 or SCD2 is below 40,0 the test is positive. However, authors do
not describe how these exact cut-offs are determined. It would be standard to use the training
cohort to find the cut-offs based on results on cases and controls, and then test the cut-offs in
the validation cohort.
3) Would authors please describe how they decide where the qPCR cut-off should be set, and
how the training and validation cohort are used in this process (if this was the case?)?
4) From what authors describe, the only difference between the validation and training cohort are
that in the training cohort the ColoDefense 1,0 and 2.0 are tested on the same samples an render
similar performance. In my opinion, this is not at training cohort, but a comparison to an already
published test in a subset of patients. Thus, it is obsolete to describe the use of ColoDefense 2.0
in the remaining patients as a validation, unless other differences characterize the analyses in
this cohort or it included another type of patients?
If this is not the case, the two cohorts should be merged to one, and the side-by-side comparison
described on a subset of patients.
Regarding the analytical performance of the ColoDefense 2.0 assay, authors have tested LoD
by a dilution series of fully methylated DNA in a background of fully unmethylated DNA.
5) Would authors please supple how high the background concentration of unmethylated DNA
was in the samples? The higher the background concentration, the higher the risk of false positive
results, that affects the assay specificity.
6) Authors also need to present data on the Limit of Blanc (LoB). In screening testing, a
negative result can be as important as a positive result and the LoB is important to ensure a
high specificity.
7) It is customary to supply the qPCR assay data and location on the DNA strand for peers to
evaluate the assay performance in their lab. Why is this data not presented?
8) Was the cut-off for a positive/negative assay set in a blinded manner or was the analyst aware
of colonoscopy results in the training and validation cohort?
I find the test on DNA degradation in the collection tubes interesting and necessary to conclude
that stool samples can be collected peripherally and analysed centrally in a reasonable work-
flow.
Results
The Results are presented as text and figures.
9) However, authors do not present how many reactions were invalid due to technical errors or
to little DNA? How about quality control indicators? How much DNA was lost in the bilsufite
conversion? How many samples were lost in the process or inconclusive? What characterized
the false positive and false negative test result individuals?
10) I find it peculiar that there is no difference in test performance across CRC stages? Usually,
the sensitivity increases with stage. Please, would authors touch upon this in the discussion?
Authors argue that the use of a single qPCR reaction (ColoDefense 2.0) compared to triplicate
reactions (ColoDefense 1,.) in the subset of patients in the training cohort does not affect the test
performance and hence it is an acceptable alternative. This could be true but is controversial, as
most literature agrees that at least two technical replicates should be made in a qPCR reaction.
11) Would authors please reflect upon why no other published PCR-based assays for CRC
detection rely on only one technical replicate. If the performance and quality is as good as authors
state, then I would expect all lab’s to only perform qPCR as single reactions.
12) A limitations section is lacking in the discussion:
Fx authors mention the need for cheap screening tools in developing countries. Is this test
comparable to the standard iFOBT/FIT test in performance and price and hence a better
alternative?
The test has not yet been applied to an actual screening cohort in a prospective trial. Is there
any plans to do this? The real performance cannot be disclosed until the test has been tried in
a proper cohort of screening patients.
assistant