Reviewer’s report  
Title: Evidence of surgical outcomes fluctuates over time: results from a cumulative meta -
analysis  of laparoscopic versus open appendec tomy for acute appendicitis  
 
Version: 0 Date: 15 Nov 2015  
 
Reviewer: Luit Penninga  
Reviewer’s report:  
The authors have written a manuscript on an interesting and importan t topic.  
 
I have the following comments:  
 
1) I agree with the authors that it is of utmost importance to recognize when firm evidence 
has been reached, and when we can stop performing randomised trials. The authors 
mention the concept of cumulative meta -analysis introduced in 1992. Since that time 
major developments have occurred and t he authors should mention these , as new 
methods look much more sophisticated on results found by meta -analyses taking into 
account diversity, risk of random errors, risk of b ias etc: fx : Sequential methods for 
random -effects meta -analysis by Higgins; Stat Med 2011, and Trial sequential analysis 
may establish when firm evidence is reached in cumulative meta -analysis by Wetterslev; 
J Clin Epidem 2008.  
 
The current manuscript mis ses this perspective, and this should be added.  
 
2) The authors write they have chosen the topic with most surgical RCTs. This is not correct 
and should be changed fx Møller et al Off -pump versus on -pump coronary artery bypass 
grafting for ischæmic Heart disease; Cochrane database of Systematic Reviews 2012, on 
Heart surgery includes 86 RCT's. Also this review by Møller includes cumulative meta -
analyses and trial sequential analysis  
 
3) A cumulative meta -analysis on the results of laparascopic vs open appe ndectomy on 
wound infection already exists; the authors should delete that they are the first ones, they 
should comment on this cumulative meta -analysis, and write that they found similar 
results for wound infection. Penninga et al; Meta -analysis of random ised trials on 
laparaoscopic versus open surgery for acute appendicitis: has firm evidence been 
reached? J Gastroint. surg 2014,  
 
4) The authors argue and assume that cumulative meta -analyses on surgical topics may 
change directions (from significant to no n-significant results) in contrast to cumulative 
meta -analyses on pharmaceutical interventions. I have seen and performed many 
cumulative meta -analyses on both surgical and pharmaceutical interventions as an editor 
for journals among others do all Cochrane  Hepato_biliary Group reviews include 
cumulative meta -analyses/trial sequntial analyses, therefore I do not agree with the 
authors. Also pharmaceutical interventions may often give 'false positive results' caused 
by small studies, with bad design, and with  low study quality. Often when larger properly 
designed trials appear significant results disappear. The authors should be more careful in 
their statement on this topic.  
 
5) Well -designed and well -performed trials may decrease risk of bias, and give more 
appropriate results. The authors should assess included trials for risk of bias, fx using the 
Cochrane risk of bias Tool. Then, the authors should perform subgroup analyses for trials 
with low and trials with high risk of bias. This will be important information  
 
Thank you  
 
Are the methods appropriate and well described?  
If not, please specify what is required in your comments to the authors.  
 
No 
 
Does the work include the necessary controls?  
If not, please spe cify which controls are required in your comments to the authors.  
 
Yes 
 
Are the conclusions drawn adequately supported by the data shown?  
If not, please explain in your comments to the authors.  
 
No 
 
Are you able to assess any statistics in the manuscript o r would you recommend an 
additional statistical review?  
If an additional statistical review is recommended, please specify what aspects require further 
assessment in your comments to the editors.  
 
I am able to assess the statistics  
 
Quality of written Engl ish 
Please indicate the quality of language in the manuscript:  
 
Acceptable  
 
Declaration of competing interests  
Please complete a declaration of competing interests, consideri ng the following questions:  
 
1. Have you in the past five years received reimbursements, fees, funding, or salary from an 
organisation that may in any way gain or lose financially from the publication of this 
manuscript, e ither now or in the future?  
 
2. Do you hold any stocks or shares in an organis ation that may in any way gain or lose 
financially from the publication of this manuscript, e ither now or in the future?  
 
3. Do you hold or are you currently applying for any patents relating to the  content of the 
manuscript?  
 
4. Have you received reimbursements , fees, funding, or salary from an organization that 
holds or has applied for patents relating to the  content of the manuscript?  
 
5. Do you have any other fin ancial competing interests?  
 
6. Do you have any non -financial competing interests  in relation to this pa per? 
 
If you can answer no to all of the above, write 'I declare that I have no competing interests' 
below. If your reply is yes to any, please give details below.  
 
I declare that I have no competing interests  
 
I agree to the open peer review policy of the  journal. I understand that my name will be included 
on my report to the authors and, if the manuscript is accepted for publication, my named report 
including any attachments I upload will be posted on the website along with the authors' 
responses. I agree  for my report to be made available under an Open Access Creative Commons 
CC-BY license ( http://creativecommons.org/licenses/by/4.0/ ). I understand that any comments 
which I do not wish to be inc luded in my named report can be included as confidential comments 
to the editors, which will not be published.  
 
I agree to the open peer review policy of the journal  
Reviewer’s report  
Title: Evidence of surgical outcomes fluctuates over time: results from a cumulative meta -
analysis  of laparoscopic versus open appendec tomy for acute appendicitis  
 
Version: 0 Date: 15 Nov 2015  
 
Reviewer: Luit Penninga  
Reviewer’s report:  
The authors have written a manuscript on an interesting and importan t topic.  
 
I have the following comments:  
 
1) I agree with the authors that it is of utmost importance to recognize when firm evidence 
has been reached, and when we can stop performing randomised trials. The authors 
mention the concept of cumulative meta -analysis introduced in 1992. Since that time 
major developments have occurred and t he authors should mention these , as new 
methods look much more sophisticated on results found by meta -analyses taking into 
account diversity, risk of random errors, risk of b ias etc: fx : Sequential methods for 
random -effects meta -analysis by Higgins; Stat Med 2011, and Trial sequential analysis 
may establish when firm evidence is reached in cumulative meta -analysis by Wetterslev; 
J Clin Epidem 2008.  
 
The current manuscript mis ses this perspective, and this should be added.  
 
2) The authors write they have chosen the topic with most surgical RCTs. This is not correct 
and should be changed fx Møller et al Off -pump versus on -pump coronary artery bypass 
grafting for ischæmic Heart disease; Cochrane database of Systematic Reviews 2012, on 
Heart surgery includes 86 RCT's. Also this review by Møller includes cumulative meta -
analyses and trial sequential analysis  
 
3) A cumulative meta -analysis on the results of laparascopic vs open appe ndectomy on 
wound infection already exists; the authors should delete that they are the first ones, they 
should comment on this cumulative meta -analysis, and write that they found similar 
results for wound infection. Penninga et al; Meta -analysis of random ised trials on 
laparaoscopic versus open surgery for acute appendicitis: has firm evidence been 
reached? J Gastroint. surg 2014,  
 
4) The authors argue and assume that cumulative meta -analyses on surgical topics may 
change directions (from significant to no n-significant results) in contrast to cumulative 
meta -analyses on pharmaceutical interventions. I have seen and performed many 
cumulative meta -analyses on both surgical and pharmaceutical interventions as an editor 
for journals among others do all Cochrane  Hepato_biliary Group reviews include 
cumulative meta -analyses/trial sequntial analyses, therefore I do not agree with the 
authors. Also pharmaceutical interventions may often give 'false positive results' caused 
by small studies, with bad design, and with  low study quality. Often when larger properly 
designed trials appear significant results disappear. The authors should be more careful in 
their statement on this topic.  
 
5) Well -designed and well -performed trials may decrease risk of bias, and give more 
appropriate results. The authors should assess included trials for risk of bias, fx using the 
Cochrane risk of bias Tool. Then, the authors should perform subgroup analyses for trials 
with low and trials with high risk of bias. This will be important information  
 
Thank you  
 
Are the methods appropriate and well described?  
If not, please specify what is required in your comments to the authors.  
 
No 
 
Does the work include the necessary controls?  
If not, please spe cify which controls are required in your comments to the authors.  
 
Yes 
 
Are the conclusions drawn adequately supported by the data shown?  
If not, please explain in your comments to the authors.  
 
No 
 
Are you able to assess any statistics in the manuscript o r would you recommend an 
additional statistical review?  
If an additional statistical review is recommended, please specify what aspects require further 
assessment in your comments to the editors.  
 
I am able to assess the statistics  
 
Quality of written Engl ish 
Please indicate the quality of language in the manuscript:  
 
Acceptable  
 
Declaration of competing interests  
Please complete a declaration of competing interests, consideri ng the following questions:  
 
1. Have you in the past five years received reimbursements, fees, funding, or salary from an 
organisation that may in any way gain or lose financially from the publication of this 
manuscript, e ither now or in the future?  
 
2. Do you hold any stocks or shares in an organis ation that may in any way gain or lose 
financially from the publication of this manuscript, e ither now or in the future?  
 
3. Do you hold or are you currently applying for any patents relating to the  content of the 
manuscript?  
 
4. Have you received reimbursements , fees, funding, or salary from an organization that 
holds or has applied for patents relating to the  content of the manuscript?  
 
5. Do you have any other fin ancial competing interests?  
 
6. Do you have any non -financial competing interests  in relation to this pa per? 
 
If you can answer no to all of the above, write 'I declare that I have no competing interests' 
below. If your reply is yes to any, please give details below.  
 
I declare that I have no competing interests  
 
I agree to the open peer review policy of the  journal. I understand that my name will be included 
on my report to the authors and, if the manuscript is accepted for publication, my named report 
including any attachments I upload will be posted on the website along with the authors' 
responses. I agree  for my report to be made available under an Open Access Creative Commons 
CC-BY license ( http://creativecommons.org/licenses/by/4.0/ ). I understand that any comments 
which I do not wish to be inc luded in my named report can be included as confidential comments 
to the editors, which will not be published.  
 
I agree to the open peer review policy of the journal  
Please review this paper.
user
Reviewer’s report  
Title: Evidence of surgical outcomes fluctuates over time: results from a cumulative meta -
analysis  of laparoscopic versus open appendec tomy for acute appendicitis  
 
Version: 0 Date: 15 Nov 2015  
 
Reviewer: Luit Penninga  
Reviewer’s report:  
The authors have written a manuscript on an interesting and importan t topic.  
 
I have the following comments:  
 
1) I agree with the authors that it is of utmost importance to recognize when firm evidence 
has been reached, and when we can stop performing randomised trials. The authors 
mention the concept of cumulative meta -analysis introduced in 1992. Since that time 
major developments have occurred and t he authors should mention these , as new 
methods look much more sophisticated on results found by meta -analyses taking into 
account diversity, risk of random errors, risk of b ias etc: fx : Sequential methods for 
random -effects meta -analysis by Higgins; Stat Med 2011, and Trial sequential analysis 
may establish when firm evidence is reached in cumulative meta -analysis by Wetterslev; 
J Clin Epidem 2008.  
 
The current manuscript mis ses this perspective, and this should be added.  
 
2) The authors write they have chosen the topic with most surgical RCTs. This is not correct 
and should be changed fx Møller et al Off -pump versus on -pump coronary artery bypass 
grafting for ischæmic Heart disease; Cochrane database of Systematic Reviews 2012, on 
Heart surgery includes 86 RCT's. Also this review by Møller includes cumulative meta -
analyses and trial sequential analysis  
 
3) A cumulative meta -analysis on the results of laparascopic vs open appe ndectomy on 
wound infection already exists; the authors should delete that they are the first ones, they 
should comment on this cumulative meta -analysis, and write that they found similar 
results for wound infection. Penninga et al; Meta -analysis of random ised trials on 
laparaoscopic versus open surgery for acute appendicitis: has firm evidence been 
reached? J Gastroint. surg 2014,  
 
4) The authors argue and assume that cumulative meta -analyses on surgical topics may 
change directions (from significant to no n-significant results) in contrast to cumulative 
meta -analyses on pharmaceutical interventions. I have seen and performed many 
cumulative meta -analyses on both surgical and pharmaceutical interventions as an editor 
for journals among others do all Cochrane  Hepato_biliary Group reviews include 
cumulative meta -analyses/trial sequntial analyses, therefore I do not agree with the 
authors. Also pharmaceutical interventions may often give 'false positive results' caused 
by small studies, with bad design, and with  low study quality. Often when larger properly 
designed trials appear significant results disappear. The authors should be more careful in 
their statement on this topic.  
 
5) Well -designed and well -performed trials may decrease risk of bias, and give more 
appropriate results. The authors should assess included trials for risk of bias, fx using the 
Cochrane risk of bias Tool. Then, the authors should perform subgroup analyses for trials 
with low and trials with high risk of bias. This will be important information  
 
Thank you  
 
Are the methods appropriate and well described?  
If not, please specify what is required in your comments to the authors.  
 
No 
 
Does the work include the necessary controls?  
If not, please spe cify which controls are required in your comments to the authors.  
 
Yes 
 
Are the conclusions drawn adequately supported by the data shown?  
If not, please explain in your comments to the authors.  
 
No 
 
Are you able to assess any statistics in the manuscript o r would you recommend an 
additional statistical review?  
If an additional statistical review is recommended, please specify what aspects require further 
assessment in your comments to the editors.  
 
I am able to assess the statistics  
 
Quality of written Engl ish 
Please indicate the quality of language in the manuscript:  
 
Acceptable  
 
Declaration of competing interests  
Please complete a declaration of competing interests, consideri ng the following questions:  
 
1. Have you in the past five years received reimbursements, fees, funding, or salary from an 
organisation that may in any way gain or lose financially from the publication of this 
manuscript, e ither now or in the future?  
 
2. Do you hold any stocks or shares in an organis ation that may in any way gain or lose 
financially from the publication of this manuscript, e ither now or in the future?  
 
3. Do you hold or are you currently applying for any patents relating to the  content of the 
manuscript?  
 
4. Have you received reimbursements , fees, funding, or salary from an organization that 
holds or has applied for patents relating to the  content of the manuscript?  
 
5. Do you have any other fin ancial competing interests?  
 
6. Do you have any non -financial competing interests  in relation to this pa per? 
 
If you can answer no to all of the above, write 'I declare that I have no competing interests' 
below. If your reply is yes to any, please give details below.  
 
I declare that I have no competing interests  
 
I agree to the open peer review policy of the  journal. I understand that my name will be included 
on my report to the authors and, if the manuscript is accepted for publication, my named report 
including any attachments I upload will be posted on the website along with the authors' 
responses. I agree  for my report to be made available under an Open Access Creative Commons 
CC-BY license ( http://creativecommons.org/licenses/by/4.0/ ). I understand that any comments 
which I do not wish to be inc luded in my named report can be included as confidential comments 
to the editors, which will not be published.  
 
I agree to the open peer review policy of the journal  
assistant