Abstract
Objectives To investigate the extent of multiplicity of results in study reports of nutrition research, and the methods specified in systematic reviews to select results for inclusion in meta-analyses.
Methods MEDLINE and Epistemonikos were searched (January 2018 – June 2019) to identify systematic reviews with meta-analysis of the association between food/diet and health-related outcomes. A random sample of these reviews was selected, and for the first presented (‘index’) meta-analysis, rules used to select effect estimates to include in this meta-analysis were extracted from the reviews and their protocols. All effect estimates from the primary studies that were eligible for inclusion in the index meta-analyses were extracted.
Results Forty-two systematic reviews were included, 14 of which had a protocol. In 29% of review protocols and 69% of reviews, at least one decision rule to select effect estimates when multiple were available was specified. In 69% (204/325) of studies included in the index meta-analyses, there was at least one type of multiplicity of results.
Conclusions Authors of systematic reviews of nutrition research should anticipate encountering multiplicity of results in the included primary studies. Specification of methods to handle multiplicity when designing reviews is therefore recommended.
What is new?
Key Findings
Key Findings
Authors of systematic reviews of nutrition research should anticipate encountering multiplicity of results in the included primary studies.
Decision rules to select results for inclusion in meta-analyses of nutrition research were infrequently pre-specified.
What this adds to what was known?
What this adds to what was known?
Previous studies have found that multiplicity of results of continuous outcomes in studies included in systematic reviews was common, and methods used to select results to include in meta-analyses were infrequently pre-specified in systematic review protocols. However, none of the studies examined meta-analyses in nutiriton research, inclusion of randomized or non-randomized studies, or where the outcome was non-continuous (e.g. binary, count or time-to-event); circumstances for which different forms of multiplicity might arise. Our study addressed this gap.
What is the implication and what should change now?
What is the implication and what should change now?
Pre-specification of decision rules to handle multiplicity when designing reviews is recommended. In the systematic review, we recommend reporting any modifications to the specified rules, or any additions that were introduced to cover multiplicity scenarios that had not been anticipated when designing the review.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This project was funded by an Australian National Health and Medical Research Council (NHMRC) project grant (APP1139997). RK is supported by a Monash Graduate Scholarship and a Monash International Tuition Scholarship. MJP is supported by an Australian Research Council Discovery Early Career Researcher Award (DE200101618). JEM is supported by an Australian NHMRC Career Development Fellowship (1143429). SM is supported by the Country Women's Association (NSW) and Edna Winifred Blackman Postgraduate Research Scholarship. The funders had no role in the study design, data collection and analysis, or preparation of the manuscript.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics approval was not required for this methodological study
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
Data are available upon request from the corresponding author.
