Abstract
Objective Quantitative bias analysis (QBA) methods evaluate the impact of biases arising from systematic errors on observational study results. This systematic review aimed to summarize the range and characteristics of quantitative bias analysis (QBA) methods for summary level data published in the peer-reviewed literature.
Study Design and Setting We searched MEDLINE, Embase, Scopus, and Web of Science for English-language articles describing QBA methods. For each QBA method, we recorded key characteristics, including applicable study designs, bias(es) addressed; bias parameters, and publicly available software. The study protocol was pre-registered on the Open Science Framework (https://osf.io/ue6vm/).
Results Our search identified 10,249 records, of which 53 were articles describing 57 QBA methods for summary level data. Of the 57 QBA methods, 51 (89%) were explicitly designed for observational studies, 2 (4%) for non-randomized interventional studies, and 4 (7%) for meta-analyses. There were 29 (51%) QBA methods that addressed unmeasured confounding, 20 (35%) misclassification bias, 5 (9%) selection bias, and 3 (5%) multiple biases. 38 (67%) QBA methods were designed to generate bias-adjusted effect estimates and 18 (32%) were designed to describe how bias could explain away observed findings. 22 (39%) articles provided code or online tools to implement the QBA methods.
Conclusion In this systematic review, we identified a total of 57 QBA methods for summary level epidemiologic data published in the peer-reviewed literature. Future investigators can use this systematic review to identify different QBA methods for summary level epidemiologic data.
Key findings This systematic review identified 57 quantitative bias analysis (QBA) methods for summary level data from observational and non-randomized interventional studies.
Overall, there were 29 QBA methods that addressed unmeasured confounding, 20 that addressed misclassification bias, 5 that addressed selection bias, and 3 that addressed multiple biases.
What this adds to what is known related to methods research within the field of clinical epidemiology?This systematic review provides an overview of the range and characteristics of QBA methods for summary level epidemiologic that are published in the peer-reviewed literature and that can be used by researchers within the field of clinical epidemiology.
What is the implication, what should change now?This systematic review may help future investigators identify different QBA methods for summary level data. However, investigators should carefully review the original manuscripts to ensure that any assumptions are fulfilled, that the necessary bias parameters are available and accurate, and that all interpretations and conclusions are made with caution.
Competing Interest Statement
In the past 36 months, TLL served as a member of the Amgen Methods Advisory Council, for which he received consulting fees and travel support. Dr Ross reported receiving grants from the US Food and Drug Administration; Johnson and Johnson; Medical Device Innovation Consortium; Agency for Healthcare Research and Quality; National Heart, Lung, and Blood Institute; and Arnold Ventures outside the submitted work. Dr Ross also is an expert witness at the request of relator attorneys, the Greene Law Firm, in a qui tam suit alleging violations of the False Claims Act and Anti-Kickback Statute against Biogen Inc. that was settled in September 2022. Dr. Jeffery reported receiving grants from the US Food and Drug Administration; Agency for Healthcare Research and Quality; National Heart, Lung, and Blood Institute; National Center for Advancing Translational Sciences; Nation Institute on Drug Abuse; and American Cancer Society. Dr. Wallach is supported by Arnold Ventures, Johnson & Johnson through the Yale Open Data Access project, and the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health under award 1K01AA028258. Dr. Wallach previously served as a consultant to Hagens Berman Sobol Shapiro LLP and Dugan Law Firm APLC.
Clinical Protocols
Funding Statement
This work was supported by the Food and Drug Administration (FDA) of the U.S. Department of Health and Human Services (HHS) as part of a financial assistance award [U01FD005938] totaling $250,000 with 100 percent funded by FDA/HHS. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by FDA/HHS, or the U.S. Government
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data produced in the present work are contained in the manuscript.
