Title: Predictors of the experience of a Cytosponge test: analysis of patient survey data from
the BEST3 trial
Version: 2 Date: 14 Sep 2022
Reviewer's report:
Thank you for sending me this manuscript. It was well written, appropriately analysed and
clearly reported. I would recommend it for publication, but make below some suggestions that
the authors may want to consider to improve their manuscript.
First, I would like to thank the authors for the clearly presented manuscript. It was a pleasure
to read, logically structured and containing all the necessary information. The analysis was well
conducted and clearly reported. The discussion was sensible and recognised the main limitations.
I have only tow main comments, regarding the adjustments in thefirst multivariable model, and
making too much of the “0.05” p-value cutoff, as explained below.
Main comments:
1)
I am not convinced by the multivariable model where all the independent variables seem to have
simply been chucked together. The point of a multivariable analysis is to get closer to a “causal”
estimate of effect, by controlling for appropriate confounders, and require some consideration.
First, your approach results in a relatively complex model (24 parameters), with only 167 cases
(=least satisfied group). Typically, it is recommended not to have more than 1 parameter for
every 10 cases. It is a rule of thumb, and I do not see any strikingly odd result here, but some of
your parameters may be unreliable because of this “overfitting”.
Second, I am not sure it makes sense to adjust the effect of the other variables for “failed
swallowing at first attempt”. Failed swallowing could well be on the causal pathway between
patients characteristics and poorer experience. For example, if males have more difficulty
swallowing, and this results in a poorer experience with Cystosponge, you do not want to control
for swallowing when looking at the effect of gender. I think the other factors are fine to be
adjusted for one another, but authors may want to think more about this.
I understand the appeal of simply fitting a model adjusted for all the variables, and I let the
authors think how they want to best address this issue.
Some possible approaches: restricting adjustment to a list of key confounders (fitting a different
model for each variable in turn), reporting multiple models (e.g. with/without swallowing), or
simply removing swallowing (they could report a separate models of the effect of swallowing
adjusted for key confounders).
2)
In your findings and conclusions, you mention several times the association between experience
and BMI. The adjusted p-value was 0.046, just below 0.05, and could really be due to chance
when looking at so many variables. It is difficult to make much more of it than “Education level”
which has a p-value of 0.061.
I understand this dichotomizing at 0.05 was pre-specified and simplify the reporting, but this
could be more nuanced in the interpretation. There seems to be a natural separation between
STAI and swallowing which are clearly associated, as opposed to alcohol, BMI, IMD etc., which
are all borderline.
Maybe both BMI and education could be reported as borderline significant, and arguably BMI
should not be included in the abstract/main findings?
Minor suggestions:
Background (page 1): Could add a few sentences on the Cystosponge procedure.
Results, Page 7 and Table 1: Could consider replacing chi-squared test by logistic regression,
as in rest of paper.
Table 2: Could merge with Table 1?
Table 1-4: Using bold in the table may put too much emphasis on the 0.05 cut-off.
Page 8, line 200: consider moving BMI sentence after anxiety and alcohol.
Page 9, line 220-224 (and discussion): mention more clearly that no strong predictor of
“difficulty swallowing” was identified.
Page 9, line 220: add p-value for gender.
Page 9, line 227-232: consider removing BMI and deprivation in the summary of findings, as
only borderline significant.
Page 10: could recognize “over fitting” and “multiple testing” in the limitations (see comments
above), and that the study was not primarily designed for this analysis
Title: Predictors of the experience of a Cytosponge test: analysis of patient survey data from
the BEST3 trial
Version: 2 Date: 14 Sep 2022
Reviewer's report:
Thank you for sending me this manuscript. It was well written, appropriately analysed and
clearly reported. I would recommend it for publication, but make below some suggestions that
the authors may want to consider to improve their manuscript.
First, I would like to thank the authors for the clearly presented manuscript. It was a pleasure
to read, logically structured and containing all the necessary information. The analysis was well
conducted and clearly reported. The discussion was sensible and recognised the main limitations.
I have only tow main comments, regarding the adjustments in thefirst multivariable model, and
making too much of the “0.05” p-value cutoff, as explained below.
Main comments:
1)
I am not convinced by the multivariable model where all the independent variables seem to have
simply been chucked together. The point of a multivariable analysis is to get closer to a “causal”
estimate of effect, by controlling for appropriate confounders, and require some consideration.
First, your approach results in a relatively complex model (24 parameters), with only 167 cases
(=least satisfied group). Typically, it is recommended not to have more than 1 parameter for
every 10 cases. It is a rule of thumb, and I do not see any strikingly odd result here, but some of
your parameters may be unreliable because of this “overfitting”.
Second, I am not sure it makes sense to adjust the effect of the other variables for “failed
swallowing at first attempt”. Failed swallowing could well be on the causal pathway between
patients characteristics and poorer experience. For example, if males have more difficulty
swallowing, and this results in a poorer experience with Cystosponge, you do not want to control
for swallowing when looking at the effect of gender. I think the other factors are fine to be
adjusted for one another, but authors may want to think more about this.
I understand the appeal of simply fitting a model adjusted for all the variables, and I let the
authors think how they want to best address this issue.
Some possible approaches: restricting adjustment to a list of key confounders (fitting a different
model for each variable in turn), reporting multiple models (e.g. with/without swallowing), or
simply removing swallowing (they could report a separate models of the effect of swallowing
adjusted for key confounders).
2)
In your findings and conclusions, you mention several times the association between experience
and BMI. The adjusted p-value was 0.046, just below 0.05, and could really be due to chance
when looking at so many variables. It is difficult to make much more of it than “Education level”
which has a p-value of 0.061.
I understand this dichotomizing at 0.05 was pre-specified and simplify the reporting, but this
could be more nuanced in the interpretation. There seems to be a natural separation between
STAI and swallowing which are clearly associated, as opposed to alcohol, BMI, IMD etc., which
are all borderline.
Maybe both BMI and education could be reported as borderline significant, and arguably BMI
should not be included in the abstract/main findings?
Minor suggestions:
Background (page 1): Could add a few sentences on the Cystosponge procedure.
Results, Page 7 and Table 1: Could consider replacing chi-squared test by logistic regression,
as in rest of paper.
Table 2: Could merge with Table 1?
Table 1-4: Using bold in the table may put too much emphasis on the 0.05 cut-off.
Page 8, line 200: consider moving BMI sentence after anxiety and alcohol.
Page 9, line 220-224 (and discussion): mention more clearly that no strong predictor of
“difficulty swallowing” was identified.
Page 9, line 220: add p-value for gender.
Page 9, line 227-232: consider removing BMI and deprivation in the summary of findings, as
only borderline significant.
Page 10: could recognize “over fitting” and “multiple testing” in the limitations (see comments
above), and that the study was not primarily designed for this analysis
Please review this paper.
user
Title: Predictors of the experience of a Cytosponge test: analysis of patient survey data from
the BEST3 trial
Version: 2 Date: 14 Sep 2022
Reviewer's report:
Thank you for sending me this manuscript. It was well written, appropriately analysed and
clearly reported. I would recommend it for publication, but make below some suggestions that
the authors may want to consider to improve their manuscript.
First, I would like to thank the authors for the clearly presented manuscript. It was a pleasure
to read, logically structured and containing all the necessary information. The analysis was well
conducted and clearly reported. The discussion was sensible and recognised the main limitations.
I have only tow main comments, regarding the adjustments in thefirst multivariable model, and
making too much of the “0.05” p-value cutoff, as explained below.
Main comments:
1)
I am not convinced by the multivariable model where all the independent variables seem to have
simply been chucked together. The point of a multivariable analysis is to get closer to a “causal”
estimate of effect, by controlling for appropriate confounders, and require some consideration.
First, your approach results in a relatively complex model (24 parameters), with only 167 cases
(=least satisfied group). Typically, it is recommended not to have more than 1 parameter for
every 10 cases. It is a rule of thumb, and I do not see any strikingly odd result here, but some of
your parameters may be unreliable because of this “overfitting”.
Second, I am not sure it makes sense to adjust the effect of the other variables for “failed
swallowing at first attempt”. Failed swallowing could well be on the causal pathway between
patients characteristics and poorer experience. For example, if males have more difficulty
swallowing, and this results in a poorer experience with Cystosponge, you do not want to control
for swallowing when looking at the effect of gender. I think the other factors are fine to be
adjusted for one another, but authors may want to think more about this.
I understand the appeal of simply fitting a model adjusted for all the variables, and I let the
authors think how they want to best address this issue.
Some possible approaches: restricting adjustment to a list of key confounders (fitting a different
model for each variable in turn), reporting multiple models (e.g. with/without swallowing), or
simply removing swallowing (they could report a separate models of the effect of swallowing
adjusted for key confounders).
2)
In your findings and conclusions, you mention several times the association between experience
and BMI. The adjusted p-value was 0.046, just below 0.05, and could really be due to chance
when looking at so many variables. It is difficult to make much more of it than “Education level”
which has a p-value of 0.061.
I understand this dichotomizing at 0.05 was pre-specified and simplify the reporting, but this
could be more nuanced in the interpretation. There seems to be a natural separation between
STAI and swallowing which are clearly associated, as opposed to alcohol, BMI, IMD etc., which
are all borderline.
Maybe both BMI and education could be reported as borderline significant, and arguably BMI
should not be included in the abstract/main findings?
Minor suggestions:
Background (page 1): Could add a few sentences on the Cystosponge procedure.
Results, Page 7 and Table 1: Could consider replacing chi-squared test by logistic regression,
as in rest of paper.
Table 2: Could merge with Table 1?
Table 1-4: Using bold in the table may put too much emphasis on the 0.05 cut-off.
Page 8, line 200: consider moving BMI sentence after anxiety and alcohol.
Page 9, line 220-224 (and discussion): mention more clearly that no strong predictor of
“difficulty swallowing” was identified.
Page 9, line 220: add p-value for gender.
Page 9, line 227-232: consider removing BMI and deprivation in the summary of findings, as
only borderline significant.
Page 10: could recognize “over fitting” and “multiple testing” in the limitations (see comments
above), and that the study was not primarily designed for this analysis
assistant