Reviewer’ s report
Title: In search for interplay between stool microRNAs, microbiota and short chain fatty acids in
Crohn’ s disease. A preliminary study.
Version: 1Date: 09 Jun 2020
Reviewer's report:
Ambrozkiewicz and colleagues have made major improvements in the content of the manuscript,
the data description and the figures presentations.However, there are still some critical points in the data analysis, that in my opinion, need to beaddressed before publication.
Major comments:1.The authors of the study cite a conveniently selected piece of the "Guidelines for Nanostring
gene Expression" to justify their lack of control for multiple testing. However, this text is taken outof its original context, which in its totality, does not support the author's justification. The citedtext refers to procedures like Bonferroni correction (which assumes independency between test) incontrast to FDR corrections procedures. In the next paragraph of the same text this becomes clear:
"In general, when multiple comparison corrections are performed on NanoString data, using a less
conservative method which allows gene significance levels to be either positively or negativelycorrelated is preferred (i.e., when probing for correlated and interdependent expression pathways).The Benjamini-Yekutieli False Discovery Rate method accounts for this expectation thatsignificant changes in genes may be correlated with or dependent on each other, and the resulting
FDR adjusted p-value provides a middle ground between the more conservative tests which
control for family-wise error rates (epitomized by tests like the Bonferroni correction) and morepermissive uncorrected p-values"
I encourage the authors to perform multiple testing correction and report both, corrected and
uncorrected p-values in the supplementary tables. Similar approach should be applied in the PLS-DA and correlation analyses. Authors can then describe in the discussion if their statisticalmethods were too conservative or too permissive. Moreover, the comparison with previouslypublished data can help the authors to discuss if their results are potentially false positive or falsenegative.
2.To reduce the number of potential false positive associations the authors could focus their
analyses in the highly prevalent OTUs in their cohort. The numbers of zeros in the top associatedfeatures in table S1, S2 and S3 denotes that these OTUs are detected only in a small subset ofsamples. For example, top signal in table S1, OTU031, is absence in the "active" group and onlypresent in 4 out of 9 controls.Zero-inflation is a common problem in microbiome studies, and together with a limited samplesize and the multiple potential confounding factors, can lead to misleading conclusions.
Alternatively, authors could perform a two-step model in which they: a) assess differences on
OTU prevalence and b) they assess differences on OTU abundances.
3.The authors should provide more details on the "Integrated analysis". It is not clear how they
have calculated the discriminatory power or if they have run any (cross-) validation test to evaluatethe model performance. Otherwise, in this case, the AUC value alone is not meaningful.Alternatively, the authors could make use of tools like LEfSe to estimate the discriminant power ofeach feature. Moreover, the authors should provide more information on the correlation betweenfeatures: type of correlation (Pearson, Spearman, others), coefficients, p-values and corrected p-
values. Scatter plots of the top correlated features will help to interpreted the results.
Other comments:
a)Were the SCFA's concentrations normalized? I suggest to perform a log10 transformation on the
values before ANOV A test and plotting.b)In order to make the introduction more consistent to the rest of the study I suggest the authors toreduce the text and link disease activity, microbiota, SCFA and miRNA to the diseasepathobiology. The current text presents each of the elements separated, and their importance in thestudy of IBD is not immediately clear.c)Could the authors indicate which was the distribution and the cut-off for the CDAI values?d)I suggest to add a "statistics analyses" section in the methods describing all computational stepsand parameters used in more details. Alternatively, authors could provide a link with all therelevant codes or commands used.e)Only one p-value is mentioned in Line 205-206: "both groups of CD patients showed loweralpha diversity"f)Line 215: "unregulated" => "upregulated". Moreover, I would suggest to replace it for
expressions like "more abundant in patients with CD".
g)Table 3 and Table 4: Are count values raw or the normalized values? Why has-miR-16-5p has aS.D. of 0 in controls? Are the lower and upper 95% C.I correct?h)P-values are not displayed in Figure 2i)Line 282. Is this reference in the correct context?j)Do the authors confirm any previously published association between taxa and SCFA levels?k)Line 370. Replace metagenomics for 16S sequencing data.l)The discussion on the miRNA is poor. I encourage the authors to expand on this part and reducethe discussion on the role of microbiome in IBD, which has already been described by others.m)Authors should also acknowledge that their findings could also be confounded by other factors,for example, all patients in remission were using biologicals or stool frequency (which was notincluded in this study).
Are the methods appropriate and well described?
If not, please specify what is required in your comments to the authors.
No
Does the work include the necessary controls?
If not, please specify which controls are required in your comments to the authors.
Unable to assessAre the conclusions drawn adequately supported by the data shown?
If not, please explain in your comments to the authors.
Unable to assess
Are you able to assess any statistics in the manuscript or would you recommend an
additional statistical review?If an additional statistical review is recommended, please specify what aspects require furtherassessment in your comments to the editors.
I am able to assess the statisticsQuality of written English
Please indicate the quality of language in the manuscript:
Needs some language corrections before being publishedDeclaration of competing interests
Please complete a declaration of competing interests, considering the following questions:
1. Have you in the past five years received reimbursements, fees, funding, or salary from an
organisation that may in any way gain or lose financially from the publication of this
manuscript, either now or in the future?
2. Do you hold any stocks or shares in an organisation that may in any way gain or lose
financially from the publication of this manuscript, either now or in the future?
3. Do you hold or are you currently applying for any patents relating to the content of the
manuscript?
4. Have you received reimbursements, fees, funding, or salary from an organization that holds
or has applied for patents relating to the content of the manuscript?
5. Do you have any other financial competing interests?6. Do you have any non-financial competing interests in relation to this paper?
If you can answer no to all of the above, write 'I declare that I have no competing interests' below.
If your reply is yes to any, please give details below.
I declare that I have no competing interests
Reviewer Publication Consent. I agree for my report to be made available under an Open Access
Creative Commons CC-BY License (http://creativecommons.org/licenses/by/4.0) if this manu-script is accepted for publication. Any comments that I do not wish to be included in the publishedreport have been included as confidential comments to the editor, which will not be published.Ifyou are not happy for us to publish this report, please contact the editorial office before completingthe review.If you wish, you can include your name in your published report. Please note you mustdecide whether to include your name at the start of the process and confirm this decision wheneverprompted. If you change your decision later, we will not be able to publish your name.
I agree to the terms of the CC-BY 4.0 license; please do not publish my name with my report.
(default)
Reviewer’ s report
Title: In search for interplay between stool microRNAs, microbiota and short chain fatty acids in
Crohn’ s disease. A preliminary study.
Version: 1Date: 09 Jun 2020
Reviewer's report:
Ambrozkiewicz and colleagues have made major improvements in the content of the manuscript,
the data description and the figures presentations.However, there are still some critical points in the data analysis, that in my opinion, need to beaddressed before publication.
Major comments:1.The authors of the study cite a conveniently selected piece of the "Guidelines for Nanostring
gene Expression" to justify their lack of control for multiple testing. However, this text is taken outof its original context, which in its totality, does not support the author's justification. The citedtext refers to procedures like Bonferroni correction (which assumes independency between test) incontrast to FDR corrections procedures. In the next paragraph of the same text this becomes clear:
"In general, when multiple comparison corrections are performed on NanoString data, using a less
conservative method which allows gene significance levels to be either positively or negativelycorrelated is preferred (i.e., when probing for correlated and interdependent expression pathways).The Benjamini-Yekutieli False Discovery Rate method accounts for this expectation thatsignificant changes in genes may be correlated with or dependent on each other, and the resulting
FDR adjusted p-value provides a middle ground between the more conservative tests which
control for family-wise error rates (epitomized by tests like the Bonferroni correction) and morepermissive uncorrected p-values"
I encourage the authors to perform multiple testing correction and report both, corrected and
uncorrected p-values in the supplementary tables. Similar approach should be applied in the PLS-DA and correlation analyses. Authors can then describe in the discussion if their statisticalmethods were too conservative or too permissive. Moreover, the comparison with previouslypublished data can help the authors to discuss if their results are potentially false positive or falsenegative.
2.To reduce the number of potential false positive associations the authors could focus their
analyses in the highly prevalent OTUs in their cohort. The numbers of zeros in the top associatedfeatures in table S1, S2 and S3 denotes that these OTUs are detected only in a small subset ofsamples. For example, top signal in table S1, OTU031, is absence in the "active" group and onlypresent in 4 out of 9 controls.Zero-inflation is a common problem in microbiome studies, and together with a limited samplesize and the multiple potential confounding factors, can lead to misleading conclusions.
Alternatively, authors could perform a two-step model in which they: a) assess differences on
OTU prevalence and b) they assess differences on OTU abundances.
3.The authors should provide more details on the "Integrated analysis". It is not clear how they
have calculated the discriminatory power or if they have run any (cross-) validation test to evaluatethe model performance. Otherwise, in this case, the AUC value alone is not meaningful.Alternatively, the authors could make use of tools like LEfSe to estimate the discriminant power ofeach feature. Moreover, the authors should provide more information on the correlation betweenfeatures: type of correlation (Pearson, Spearman, others), coefficients, p-values and corrected p-
values. Scatter plots of the top correlated features will help to interpreted the results.
Other comments:
a)Were the SCFA's concentrations normalized? I suggest to perform a log10 transformation on the
values before ANOV A test and plotting.b)In order to make the introduction more consistent to the rest of the study I suggest the authors toreduce the text and link disease activity, microbiota, SCFA and miRNA to the diseasepathobiology. The current text presents each of the elements separated, and their importance in thestudy of IBD is not immediately clear.c)Could the authors indicate which was the distribution and the cut-off for the CDAI values?d)I suggest to add a "statistics analyses" section in the methods describing all computational stepsand parameters used in more details. Alternatively, authors could provide a link with all therelevant codes or commands used.e)Only one p-value is mentioned in Line 205-206: "both groups of CD patients showed loweralpha diversity"f)Line 215: "unregulated" => "upregulated". Moreover, I would suggest to replace it for
expressions like "more abundant in patients with CD".
g)Table 3 and Table 4: Are count values raw or the normalized values? Why has-miR-16-5p has aS.D. of 0 in controls? Are the lower and upper 95% C.I correct?h)P-values are not displayed in Figure 2i)Line 282. Is this reference in the correct context?j)Do the authors confirm any previously published association between taxa and SCFA levels?k)Line 370. Replace metagenomics for 16S sequencing data.l)The discussion on the miRNA is poor. I encourage the authors to expand on this part and reducethe discussion on the role of microbiome in IBD, which has already been described by others.m)Authors should also acknowledge that their findings could also be confounded by other factors,for example, all patients in remission were using biologicals or stool frequency (which was notincluded in this study).
Are the methods appropriate and well described?
If not, please specify what is required in your comments to the authors.
No
Does the work include the necessary controls?
If not, please specify which controls are required in your comments to the authors.
Unable to assessAre the conclusions drawn adequately supported by the data shown?
If not, please explain in your comments to the authors.
Unable to assess
Are you able to assess any statistics in the manuscript or would you recommend an
additional statistical review?If an additional statistical review is recommended, please specify what aspects require furtherassessment in your comments to the editors.
I am able to assess the statisticsQuality of written English
Please indicate the quality of language in the manuscript:
Needs some language corrections before being publishedDeclaration of competing interests
Please complete a declaration of competing interests, considering the following questions:
1. Have you in the past five years received reimbursements, fees, funding, or salary from an
organisation that may in any way gain or lose financially from the publication of this
manuscript, either now or in the future?
2. Do you hold any stocks or shares in an organisation that may in any way gain or lose
financially from the publication of this manuscript, either now or in the future?
3. Do you hold or are you currently applying for any patents relating to the content of the
manuscript?
4. Have you received reimbursements, fees, funding, or salary from an organization that holds
or has applied for patents relating to the content of the manuscript?
5. Do you have any other financial competing interests?6. Do you have any non-financial competing interests in relation to this paper?
If you can answer no to all of the above, write 'I declare that I have no competing interests' below.
If your reply is yes to any, please give details below.
I declare that I have no competing interests
Reviewer Publication Consent. I agree for my report to be made available under an Open Access
Creative Commons CC-BY License (http://creativecommons.org/licenses/by/4.0) if this manu-script is accepted for publication. Any comments that I do not wish to be included in the publishedreport have been included as confidential comments to the editor, which will not be published.Ifyou are not happy for us to publish this report, please contact the editorial office before completingthe review.If you wish, you can include your name in your published report. Please note you mustdecide whether to include your name at the start of the process and confirm this decision wheneverprompted. If you change your decision later, we will not be able to publish your name.
I agree to the terms of the CC-BY 4.0 license; please do not publish my name with my report.
(default)
Please review this paper.
user
Reviewer’ s report
Title: In search for interplay between stool microRNAs, microbiota and short chain fatty acids in
Crohn’ s disease. A preliminary study.
Version: 1Date: 09 Jun 2020
Reviewer's report:
Ambrozkiewicz and colleagues have made major improvements in the content of the manuscript,
the data description and the figures presentations.However, there are still some critical points in the data analysis, that in my opinion, need to beaddressed before publication.
Major comments:1.The authors of the study cite a conveniently selected piece of the "Guidelines for Nanostring
gene Expression" to justify their lack of control for multiple testing. However, this text is taken outof its original context, which in its totality, does not support the author's justification. The citedtext refers to procedures like Bonferroni correction (which assumes independency between test) incontrast to FDR corrections procedures. In the next paragraph of the same text this becomes clear:
"In general, when multiple comparison corrections are performed on NanoString data, using a less
conservative method which allows gene significance levels to be either positively or negativelycorrelated is preferred (i.e., when probing for correlated and interdependent expression pathways).The Benjamini-Yekutieli False Discovery Rate method accounts for this expectation thatsignificant changes in genes may be correlated with or dependent on each other, and the resulting
FDR adjusted p-value provides a middle ground between the more conservative tests which
control for family-wise error rates (epitomized by tests like the Bonferroni correction) and morepermissive uncorrected p-values"
I encourage the authors to perform multiple testing correction and report both, corrected and
uncorrected p-values in the supplementary tables. Similar approach should be applied in the PLS-DA and correlation analyses. Authors can then describe in the discussion if their statisticalmethods were too conservative or too permissive. Moreover, the comparison with previouslypublished data can help the authors to discuss if their results are potentially false positive or falsenegative.
2.To reduce the number of potential false positive associations the authors could focus their
analyses in the highly prevalent OTUs in their cohort. The numbers of zeros in the top associatedfeatures in table S1, S2 and S3 denotes that these OTUs are detected only in a small subset ofsamples. For example, top signal in table S1, OTU031, is absence in the "active" group and onlypresent in 4 out of 9 controls.Zero-inflation is a common problem in microbiome studies, and together with a limited samplesize and the multiple potential confounding factors, can lead to misleading conclusions.
Alternatively, authors could perform a two-step model in which they: a) assess differences on
OTU prevalence and b) they assess differences on OTU abundances.
3.The authors should provide more details on the "Integrated analysis". It is not clear how they
have calculated the discriminatory power or if they have run any (cross-) validation test to evaluatethe model performance. Otherwise, in this case, the AUC value alone is not meaningful.Alternatively, the authors could make use of tools like LEfSe to estimate the discriminant power ofeach feature. Moreover, the authors should provide more information on the correlation betweenfeatures: type of correlation (Pearson, Spearman, others), coefficients, p-values and corrected p-
values. Scatter plots of the top correlated features will help to interpreted the results.
Other comments:
a)Were the SCFA's concentrations normalized? I suggest to perform a log10 transformation on the
values before ANOV A test and plotting.b)In order to make the introduction more consistent to the rest of the study I suggest the authors toreduce the text and link disease activity, microbiota, SCFA and miRNA to the diseasepathobiology. The current text presents each of the elements separated, and their importance in thestudy of IBD is not immediately clear.c)Could the authors indicate which was the distribution and the cut-off for the CDAI values?d)I suggest to add a "statistics analyses" section in the methods describing all computational stepsand parameters used in more details. Alternatively, authors could provide a link with all therelevant codes or commands used.e)Only one p-value is mentioned in Line 205-206: "both groups of CD patients showed loweralpha diversity"f)Line 215: "unregulated" => "upregulated". Moreover, I would suggest to replace it for
expressions like "more abundant in patients with CD".
g)Table 3 and Table 4: Are count values raw or the normalized values? Why has-miR-16-5p has aS.D. of 0 in controls? Are the lower and upper 95% C.I correct?h)P-values are not displayed in Figure 2i)Line 282. Is this reference in the correct context?j)Do the authors confirm any previously published association between taxa and SCFA levels?k)Line 370. Replace metagenomics for 16S sequencing data.l)The discussion on the miRNA is poor. I encourage the authors to expand on this part and reducethe discussion on the role of microbiome in IBD, which has already been described by others.m)Authors should also acknowledge that their findings could also be confounded by other factors,for example, all patients in remission were using biologicals or stool frequency (which was notincluded in this study).
Are the methods appropriate and well described?
If not, please specify what is required in your comments to the authors.
No
Does the work include the necessary controls?
If not, please specify which controls are required in your comments to the authors.
Unable to assessAre the conclusions drawn adequately supported by the data shown?
If not, please explain in your comments to the authors.
Unable to assess
Are you able to assess any statistics in the manuscript or would you recommend an
additional statistical review?If an additional statistical review is recommended, please specify what aspects require furtherassessment in your comments to the editors.
I am able to assess the statisticsQuality of written English
Please indicate the quality of language in the manuscript:
Needs some language corrections before being publishedDeclaration of competing interests
Please complete a declaration of competing interests, considering the following questions:
1. Have you in the past five years received reimbursements, fees, funding, or salary from an
organisation that may in any way gain or lose financially from the publication of this
manuscript, either now or in the future?
2. Do you hold any stocks or shares in an organisation that may in any way gain or lose
financially from the publication of this manuscript, either now or in the future?
3. Do you hold or are you currently applying for any patents relating to the content of the
manuscript?
4. Have you received reimbursements, fees, funding, or salary from an organization that holds
or has applied for patents relating to the content of the manuscript?
5. Do you have any other financial competing interests?6. Do you have any non-financial competing interests in relation to this paper?
If you can answer no to all of the above, write 'I declare that I have no competing interests' below.
If your reply is yes to any, please give details below.
I declare that I have no competing interests
Reviewer Publication Consent. I agree for my report to be made available under an Open Access
Creative Commons CC-BY License (http://creativecommons.org/licenses/by/4.0) if this manu-script is accepted for publication. Any comments that I do not wish to be included in the publishedreport have been included as confidential comments to the editor, which will not be published.Ifyou are not happy for us to publish this report, please contact the editorial office before completingthe review.If you wish, you can include your name in your published report. Please note you mustdecide whether to include your name at the start of the process and confirm this decision wheneverprompted. If you change your decision later, we will not be able to publish your name.
I agree to the terms of the CC-BY 4.0 license; please do not publish my name with my report.
(default)
assistant