Reviewer's report
Title:Endoscopist specialty is associated with colonoscopy quality
Version: 2Date:25 November 2012
Reviewer: Salah Mahmud
Reviewer's report:
The authors are to be commended for a well-written and interesting manuscript. I
hope that the following comments can be of some assistance in improving the
manuscript.
Abstract:
- I would recommend defining the study’s main outcome “polypectomy rate”
(more about this below).
- To facilitate the interpretation of the adjusted ORs, please clarify what speciality
was used as the reference group.
Methods:
- The authors are clearly aware of the limitations of administrative databases for
the identification of polypectomies (e.g., up to 15% of polypectomies may not be
recorded, ref #18). The authors attempted to minimize the resulting
misclassification by using overlapping data sources (in Calgary) and via model
adjustments. This raises several questions:
o First, why was information on polypectomy (and other important quality
indicators, e.g., adenoma detection) not gathered by the participating
endoscopists the same way they gathered information on patient characteristics?
I would imagine that most endoscopists routinely document at least some of that
information (e.g., the presence of polyps or the performance of polypectomy) for
their own use (e.g., medico-legal purposes or to secure reimbursement for
fee-for-service physicians). Gathering this information would have permitted the
authors to validate the use of administrative databases (for future studies where
only administrative data is available), avoid misclassification of polypectomy
status and directly assess the effect of specialty on adenoma detection rates
(which they use the polypectomy rate as a proxy for).
o It is not clear how the abovementioned model adjustment for polypectomy
misclassification was carried out, so it is not possible to assess whether the
adjustment accounts for the fact that misclassification (due to underreporting of
polypectomies) is differential, which could occur if for example the surgeons are
equally effective at removing polyps, but they are less likely to bill for them?
- I would recommend defining the outcome “polypectomy rate.” I suspect that the
term rate is not used in its strict epidemiologic sense, because if the outcome
was indeed a rate, the authors would probably have opted to use a Poisson or
negative binomial model rather than a logistic regression model. Their use of the
logistic regression model suggests that the outcome was the probability of
identifying a polypectomy for an eligible patient who had undergone scoping.
- Were there patients who had more than one scope during the study period? If
so, how they were handled in the multi-level analysis?
Discussion:
- The authors are aware of the potential for confounding by patient-level
(case-mix) factors. I would have liked to have seen a similar consideration given
to endoscopist-level factors, e.g., endoscopist volume and institutional factors
(e.g., whether an endoscopist’s main practice is located in tertiary/teaching
hospital). I am aware that the small number of endoscopists would have limited
the authors’ ability to assess for confounding or interactions by these factors, but
the possibility of confounding should be at least mentioned (and its potential
impact is assessed) in the discussion. Accounting for these factors may have
also explained some of the significant variability between individual endoscopists
after accounting for specialty and case-mix factors.
- In my opinion, there are two main difficulties in interpreting the results of this,
and other similar studies, which could potentially limit the value of this line of
enquiry:
- First, like most previous studies in the literature, this study is limited by the
small sample size of endoscopists, especially those who perform a meaningful
number of endoscopies on a regular basis. (I do not think the statistical ‘noise’
generated by including physicians who perform 2 or 3 endoscopies every other
year justifies the increase in statistical power.) As a result, these studies are
generally limited in their ability to assess the likely complex interactions between
endoscopist characteristics such as volume, specialty, site of practice,
experience, training, and local patterns of referral. For example, in the current
study it is not possible, based on the presented evidence, to conclude whether
the higher average polypectomy rates among gastroenterologists is due to
factors associated with their specialization (e.g., training) or to the fact that they,
on average, perform more colonoscopies. This is an important consideration from
a policy point of view. If volume is more important than specialty, then a policy of
discouraging low-volume practice will be more appropriate than (for instance) a
policy based on discouraging certain specialities from performing endoscopies.
- More importantly, the present study, like many other similar studies, do not
attempt to explain the technical factors that would explain why
gastroenterologists (or high-volume or experienced endoscopists) have a higher
polypectomy rate, or a lower “missed” CRC rates. Is this related to better patient
preparation, more effective sedation, personal skills, etc.? Answering these
questions is an essential step for improving the quality of endoscopies performed
by all endoscopists.
Thanks,
Level of interest: An article of importance in its field
Quality of written English: Acceptable
Statistical review: Yes, and I have assessed the statistics in my report.
Declaration of competing interests:
I declare that I have no competing interests
Reviewer's report
Title:Endoscopist specialty is associated with colonoscopy quality
Version: 2Date:25 November 2012
Reviewer: Salah Mahmud
Reviewer's report:
The authors are to be commended for a well-written and interesting manuscript. I
hope that the following comments can be of some assistance in improving the
manuscript.
Abstract:
- I would recommend defining the study’s main outcome “polypectomy rate”
(more about this below).
- To facilitate the interpretation of the adjusted ORs, please clarify what speciality
was used as the reference group.
Methods:
- The authors are clearly aware of the limitations of administrative databases for
the identification of polypectomies (e.g., up to 15% of polypectomies may not be
recorded, ref #18). The authors attempted to minimize the resulting
misclassification by using overlapping data sources (in Calgary) and via model
adjustments. This raises several questions:
o First, why was information on polypectomy (and other important quality
indicators, e.g., adenoma detection) not gathered by the participating
endoscopists the same way they gathered information on patient characteristics?
I would imagine that most endoscopists routinely document at least some of that
information (e.g., the presence of polyps or the performance of polypectomy) for
their own use (e.g., medico-legal purposes or to secure reimbursement for
fee-for-service physicians). Gathering this information would have permitted the
authors to validate the use of administrative databases (for future studies where
only administrative data is available), avoid misclassification of polypectomy
status and directly assess the effect of specialty on adenoma detection rates
(which they use the polypectomy rate as a proxy for).
o It is not clear how the abovementioned model adjustment for polypectomy
misclassification was carried out, so it is not possible to assess whether the
adjustment accounts for the fact that misclassification (due to underreporting of
polypectomies) is differential, which could occur if for example the surgeons are
equally effective at removing polyps, but they are less likely to bill for them?
- I would recommend defining the outcome “polypectomy rate.” I suspect that the
term rate is not used in its strict epidemiologic sense, because if the outcome
was indeed a rate, the authors would probably have opted to use a Poisson or
negative binomial model rather than a logistic regression model. Their use of the
logistic regression model suggests that the outcome was the probability of
identifying a polypectomy for an eligible patient who had undergone scoping.
- Were there patients who had more than one scope during the study period? If
so, how they were handled in the multi-level analysis?
Discussion:
- The authors are aware of the potential for confounding by patient-level
(case-mix) factors. I would have liked to have seen a similar consideration given
to endoscopist-level factors, e.g., endoscopist volume and institutional factors
(e.g., whether an endoscopist’s main practice is located in tertiary/teaching
hospital). I am aware that the small number of endoscopists would have limited
the authors’ ability to assess for confounding or interactions by these factors, but
the possibility of confounding should be at least mentioned (and its potential
impact is assessed) in the discussion. Accounting for these factors may have
also explained some of the significant variability between individual endoscopists
after accounting for specialty and case-mix factors.
- In my opinion, there are two main difficulties in interpreting the results of this,
and other similar studies, which could potentially limit the value of this line of
enquiry:
- First, like most previous studies in the literature, this study is limited by the
small sample size of endoscopists, especially those who perform a meaningful
number of endoscopies on a regular basis. (I do not think the statistical ‘noise’
generated by including physicians who perform 2 or 3 endoscopies every other
year justifies the increase in statistical power.) As a result, these studies are
generally limited in their ability to assess the likely complex interactions between
endoscopist characteristics such as volume, specialty, site of practice,
experience, training, and local patterns of referral. For example, in the current
study it is not possible, based on the presented evidence, to conclude whether
the higher average polypectomy rates among gastroenterologists is due to
factors associated with their specialization (e.g., training) or to the fact that they,
on average, perform more colonoscopies. This is an important consideration from
a policy point of view. If volume is more important than specialty, then a policy of
discouraging low-volume practice will be more appropriate than (for instance) a
policy based on discouraging certain specialities from performing endoscopies.
- More importantly, the present study, like many other similar studies, do not
attempt to explain the technical factors that would explain why
gastroenterologists (or high-volume or experienced endoscopists) have a higher
polypectomy rate, or a lower “missed” CRC rates. Is this related to better patient
preparation, more effective sedation, personal skills, etc.? Answering these
questions is an essential step for improving the quality of endoscopies performed
by all endoscopists.
Thanks,
Level of interest: An article of importance in its field
Quality of written English: Acceptable
Statistical review: Yes, and I have assessed the statistics in my report.
Declaration of competing interests:
I declare that I have no competing interests
Please review this paper.
user
Reviewer's report
Title:Endoscopist specialty is associated with colonoscopy quality
Version: 2Date:25 November 2012
Reviewer: Salah Mahmud
Reviewer's report:
The authors are to be commended for a well-written and interesting manuscript. I
hope that the following comments can be of some assistance in improving the
manuscript.
Abstract:
- I would recommend defining the study’s main outcome “polypectomy rate”
(more about this below).
- To facilitate the interpretation of the adjusted ORs, please clarify what speciality
was used as the reference group.
Methods:
- The authors are clearly aware of the limitations of administrative databases for
the identification of polypectomies (e.g., up to 15% of polypectomies may not be
recorded, ref #18). The authors attempted to minimize the resulting
misclassification by using overlapping data sources (in Calgary) and via model
adjustments. This raises several questions:
o First, why was information on polypectomy (and other important quality
indicators, e.g., adenoma detection) not gathered by the participating
endoscopists the same way they gathered information on patient characteristics?
I would imagine that most endoscopists routinely document at least some of that
information (e.g., the presence of polyps or the performance of polypectomy) for
their own use (e.g., medico-legal purposes or to secure reimbursement for
fee-for-service physicians). Gathering this information would have permitted the
authors to validate the use of administrative databases (for future studies where
only administrative data is available), avoid misclassification of polypectomy
status and directly assess the effect of specialty on adenoma detection rates
(which they use the polypectomy rate as a proxy for).
o It is not clear how the abovementioned model adjustment for polypectomy
misclassification was carried out, so it is not possible to assess whether the
adjustment accounts for the fact that misclassification (due to underreporting of
polypectomies) is differential, which could occur if for example the surgeons are
equally effective at removing polyps, but they are less likely to bill for them?
- I would recommend defining the outcome “polypectomy rate.” I suspect that the
term rate is not used in its strict epidemiologic sense, because if the outcome
was indeed a rate, the authors would probably have opted to use a Poisson or
negative binomial model rather than a logistic regression model. Their use of the
logistic regression model suggests that the outcome was the probability of
identifying a polypectomy for an eligible patient who had undergone scoping.
- Were there patients who had more than one scope during the study period? If
so, how they were handled in the multi-level analysis?
Discussion:
- The authors are aware of the potential for confounding by patient-level
(case-mix) factors. I would have liked to have seen a similar consideration given
to endoscopist-level factors, e.g., endoscopist volume and institutional factors
(e.g., whether an endoscopist’s main practice is located in tertiary/teaching
hospital). I am aware that the small number of endoscopists would have limited
the authors’ ability to assess for confounding or interactions by these factors, but
the possibility of confounding should be at least mentioned (and its potential
impact is assessed) in the discussion. Accounting for these factors may have
also explained some of the significant variability between individual endoscopists
after accounting for specialty and case-mix factors.
- In my opinion, there are two main difficulties in interpreting the results of this,
and other similar studies, which could potentially limit the value of this line of
enquiry:
- First, like most previous studies in the literature, this study is limited by the
small sample size of endoscopists, especially those who perform a meaningful
number of endoscopies on a regular basis. (I do not think the statistical ‘noise’
generated by including physicians who perform 2 or 3 endoscopies every other
year justifies the increase in statistical power.) As a result, these studies are
generally limited in their ability to assess the likely complex interactions between
endoscopist characteristics such as volume, specialty, site of practice,
experience, training, and local patterns of referral. For example, in the current
study it is not possible, based on the presented evidence, to conclude whether
the higher average polypectomy rates among gastroenterologists is due to
factors associated with their specialization (e.g., training) or to the fact that they,
on average, perform more colonoscopies. This is an important consideration from
a policy point of view. If volume is more important than specialty, then a policy of
discouraging low-volume practice will be more appropriate than (for instance) a
policy based on discouraging certain specialities from performing endoscopies.
- More importantly, the present study, like many other similar studies, do not
attempt to explain the technical factors that would explain why
gastroenterologists (or high-volume or experienced endoscopists) have a higher
polypectomy rate, or a lower “missed” CRC rates. Is this related to better patient
preparation, more effective sedation, personal skills, etc.? Answering these
questions is an essential step for improving the quality of endoscopies performed
by all endoscopists.
Thanks,
Level of interest: An article of importance in its field
Quality of written English: Acceptable
Statistical review: Yes, and I have assessed the statistics in my report.
Declaration of competing interests:
I declare that I have no competing interests
assistant