Title: Comparison of different second line treatments for metastatic pancreatic cancer: a
systematic review and network meta-analysis
Version: 0 Date: 08 Jan 2023
Reviewer's report:
Petrelli et al conducted a systematic review and network meta-analysis in an effort to synthesize
available evidence in the space of 2L metastatic pancreatic ductal adenocarcinoma. Please find
my comments below for major revisions:
Abstract:
1.  In methods, the authors state that "We performed an NMA with hazard ratios (HRs) and 95%-
confidence intervals (CIs) to evaluate comparative effectiveness among different second-line
therapies in mPDAC".  Were the HRs precomputed? Also, the authors state confidence intervals
in the abstract while stated Bayesian framework in the main text which revolves around credible
intervals.
2. In methods, author stated that P-scores were used. P-scores are measures to assess relative
treatment rankings within the Frequentist framework and not within the Bayesian framework.
The SUCRA scores are analogous to P-scores in Bayesian analysis
3. In methods, the authors only state OS as the primary outcome but presented results for PFS
too. It becomes problematic to think of endpoints as primary or secondary in a meta-analysis.
Phrases like "main outcomes of interest" should be used and should state all outcomes assessed.
4. In results, the authors should present the total number of trials with the total number patients
included in the analysis to provide a general sense of the reliability of the results.
5.  In results, the author stated, "As for PFS NMA, NALIRI + 5-FU and folinic acid trended
to best treatment compared to all other regimens (P-score= 0.55) and ranked to be the best
treatment". The authors appear to be definitive in their statements; the P-score was only 0.55
reflecting unremarkable uncertainty around the relative treatment effect estimates. Caution is
warranted and findings should be suggestive; terms such as 'may', 'potentially' should be used
for these findings in both, the results and the conclusion sections.
Main Text:
1.  Systematic reviews are reported in accordance with the PRISMA statement and not
performed. I'd consider re-phrasing the statement.
2. The eligibility criteria appear to be vague. More clarity is needed. Were phase 4 RCTs
included? Were abstracts included?
3. The author stated the use of Bayesian framework for NMA. The lack of convergence in
Bayesian analysis is a major issue. How was convergence assessed? Did the model converge?
Please provide statistics and graphs for convergence as supplementary appendices. P-scores
should be removed as the term is relevant to Frequentist NMA
4. The statement, "Treatments with the highest and lowest p-scores are considered the best
and worst treatments, respectively" is misleading. Ranking scores in NMA are not reliable and
should only be interpretated in congruency with pairwise estimates. I'd consider cumulative
scores graph or table and not the probability matrix (Table 3).
5. In limitations, add small number of trials, and variable follow up durations of the trials
included.
6. This is no risk of bias assessment. Need to present the quality of trials included in this analysis
considering that there are some concerns around the trials included owing to premature closure/
termination.
7. General, the whole tone of the manuscript needs to be suggestive instead of definitive. While
the authors did a great job organizing data, the evidence is not substantial, and the findings are
still sparse and imprecise. Hence, the certainty of evidence is low. This is something that the
authors fail to discuss in great detail in the discussion section.
Title: Comparison of different second line treatments for metastatic pancreatic cancer: a
systematic review and network meta-analysis
Version: 0 Date: 08 Jan 2023
Reviewer's report:
Petrelli et al conducted a systematic review and network meta-analysis in an effort to synthesize
available evidence in the space of 2L metastatic pancreatic ductal adenocarcinoma. Please find
my comments below for major revisions:
Abstract:
1.  In methods, the authors state that "We performed an NMA with hazard ratios (HRs) and 95%-
confidence intervals (CIs) to evaluate comparative effectiveness among different second-line
therapies in mPDAC".  Were the HRs precomputed? Also, the authors state confidence intervals
in the abstract while stated Bayesian framework in the main text which revolves around credible
intervals.
2. In methods, author stated that P-scores were used. P-scores are measures to assess relative
treatment rankings within the Frequentist framework and not within the Bayesian framework.
The SUCRA scores are analogous to P-scores in Bayesian analysis
3. In methods, the authors only state OS as the primary outcome but presented results for PFS
too. It becomes problematic to think of endpoints as primary or secondary in a meta-analysis.
Phrases like "main outcomes of interest" should be used and should state all outcomes assessed.
4. In results, the authors should present the total number of trials with the total number patients
included in the analysis to provide a general sense of the reliability of the results.
5.  In results, the author stated, "As for PFS NMA, NALIRI + 5-FU and folinic acid trended
to best treatment compared to all other regimens (P-score= 0.55) and ranked to be the best
treatment". The authors appear to be definitive in their statements; the P-score was only 0.55
reflecting unremarkable uncertainty around the relative treatment effect estimates. Caution is
warranted and findings should be suggestive; terms such as 'may', 'potentially' should be used
for these findings in both, the results and the conclusion sections.
Main Text:
1.  Systematic reviews are reported in accordance with the PRISMA statement and not
performed. I'd consider re-phrasing the statement.
2. The eligibility criteria appear to be vague. More clarity is needed. Were phase 4 RCTs
included? Were abstracts included?
3. The author stated the use of Bayesian framework for NMA. The lack of convergence in
Bayesian analysis is a major issue. How was convergence assessed? Did the model converge?
Please provide statistics and graphs for convergence as supplementary appendices. P-scores
should be removed as the term is relevant to Frequentist NMA
4. The statement, "Treatments with the highest and lowest p-scores are considered the best
and worst treatments, respectively" is misleading. Ranking scores in NMA are not reliable and
should only be interpretated in congruency with pairwise estimates. I'd consider cumulative
scores graph or table and not the probability matrix (Table 3).
5. In limitations, add small number of trials, and variable follow up durations of the trials
included.
6. This is no risk of bias assessment. Need to present the quality of trials included in this analysis
considering that there are some concerns around the trials included owing to premature closure/
termination.
7. General, the whole tone of the manuscript needs to be suggestive instead of definitive. While
the authors did a great job organizing data, the evidence is not substantial, and the findings are
still sparse and imprecise. Hence, the certainty of evidence is low. This is something that the
authors fail to discuss in great detail in the discussion section.
Please review this paper.
user
Title: Comparison of different second line treatments for metastatic pancreatic cancer: a
systematic review and network meta-analysis
Version: 0 Date: 08 Jan 2023
Reviewer's report:
Petrelli et al conducted a systematic review and network meta-analysis in an effort to synthesize
available evidence in the space of 2L metastatic pancreatic ductal adenocarcinoma. Please find
my comments below for major revisions:
Abstract:
1.  In methods, the authors state that "We performed an NMA with hazard ratios (HRs) and 95%-
confidence intervals (CIs) to evaluate comparative effectiveness among different second-line
therapies in mPDAC".  Were the HRs precomputed? Also, the authors state confidence intervals
in the abstract while stated Bayesian framework in the main text which revolves around credible
intervals.
2. In methods, author stated that P-scores were used. P-scores are measures to assess relative
treatment rankings within the Frequentist framework and not within the Bayesian framework.
The SUCRA scores are analogous to P-scores in Bayesian analysis
3. In methods, the authors only state OS as the primary outcome but presented results for PFS
too. It becomes problematic to think of endpoints as primary or secondary in a meta-analysis.
Phrases like "main outcomes of interest" should be used and should state all outcomes assessed.
4. In results, the authors should present the total number of trials with the total number patients
included in the analysis to provide a general sense of the reliability of the results.
5.  In results, the author stated, "As for PFS NMA, NALIRI + 5-FU and folinic acid trended
to best treatment compared to all other regimens (P-score= 0.55) and ranked to be the best
treatment". The authors appear to be definitive in their statements; the P-score was only 0.55
reflecting unremarkable uncertainty around the relative treatment effect estimates. Caution is
warranted and findings should be suggestive; terms such as 'may', 'potentially' should be used
for these findings in both, the results and the conclusion sections.
Main Text:
1.  Systematic reviews are reported in accordance with the PRISMA statement and not
performed. I'd consider re-phrasing the statement.
2. The eligibility criteria appear to be vague. More clarity is needed. Were phase 4 RCTs
included? Were abstracts included?
3. The author stated the use of Bayesian framework for NMA. The lack of convergence in
Bayesian analysis is a major issue. How was convergence assessed? Did the model converge?
Please provide statistics and graphs for convergence as supplementary appendices. P-scores
should be removed as the term is relevant to Frequentist NMA
4. The statement, "Treatments with the highest and lowest p-scores are considered the best
and worst treatments, respectively" is misleading. Ranking scores in NMA are not reliable and
should only be interpretated in congruency with pairwise estimates. I'd consider cumulative
scores graph or table and not the probability matrix (Table 3).
5. In limitations, add small number of trials, and variable follow up durations of the trials
included.
6. This is no risk of bias assessment. Need to present the quality of trials included in this analysis
considering that there are some concerns around the trials included owing to premature closure/
termination.
7. General, the whole tone of the manuscript needs to be suggestive instead of definitive. While
the authors did a great job organizing data, the evidence is not substantial, and the findings are
still sparse and imprecise. Hence, the certainty of evidence is low. This is something that the
authors fail to discuss in great detail in the discussion section.
assistant