Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Quantitative bias analysis methods for summary level epidemiologic data in the peer-reviewed literature: a systematic review

View ORCID ProfileXiaoting Shi, Ziang Liu, Mingfeng Zhang, Wei Hua, Jie Li, Joo-Yeon Lee, Sai Dharmarajan, Kate Nyhan, Ashley Naimi, Timothy L. Lash, Molly M. Jeffery, View ORCID ProfileJoseph S. Ross, Zeyan Liew, View ORCID ProfileJoshua D. Wallach
doi: https://doi.org/10.1101/2024.04.23.24306205
Xiaoting Shi
1Department of Environmental Health Sciences, Yale School of Public Health, New Haven, Connecticut, USA
PhD, MPhil
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Xiaoting Shi
Ziang Liu
2Department of Epidemiology and Biostatistics, Arnold School of Public Health, University of South Carolina, Columbia, South Carolina, USA
MS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mingfeng Zhang
3Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Wei Hua
3Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jie Li
3Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Joo-Yeon Lee
4Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, USA Food and Drug Administration, Silver Spring, Maryland, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sai Dharmarajan
5Sarepta Therapeutics, Cambridge, Massachusetts, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kate Nyhan
1Department of Environmental Health Sciences, Yale School of Public Health, New Haven, Connecticut, USA
6Cushing/Whitney Medical Library, Yale University, New Haven, Connecticut, USA
MLS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ashley Naimi
7Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Timothy L. Lash
7Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia
DSc, MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Molly M. Jeffery
8Division of Health Care Delivery Research, Mayo Clinic, Rochester, Minnesota, USA
9Division of Emergency Medicine, Mayo Clinic, Rochester, Minnesota, USA
PhD MPP
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Joseph S. Ross
10Section of General Medicine and the National Clinician Scholars Program, Department of Internal Medicine, Yale University School of Medicine, New Haven, Connecticut, USA
11Department of Health Policy and Management, Yale School of Public Health, New Haven, Connecticut, USA
12Center for Outcomes Research and Evaluation, Yale-New Haven Health. New Haven, Connecticut, USA
MD, MHS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Joseph S. Ross
Zeyan Liew
1Department of Environmental Health Sciences, Yale School of Public Health, New Haven, Connecticut, USA
PhD, MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Joshua D. Wallach
7Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia
PhD, MS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Joshua D. Wallach
  • For correspondence: joshua.wallach{at}emory.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Objective Quantitative bias analysis (QBA) methods evaluate the impact of biases arising from systematic errors on observational study results. This systematic review aimed to summarize the range and characteristics of quantitative bias analysis (QBA) methods for summary level data published in the peer-reviewed literature.

Study Design and Setting We searched MEDLINE, Embase, Scopus, and Web of Science for English-language articles describing QBA methods. For each QBA method, we recorded key characteristics, including applicable study designs, bias(es) addressed; bias parameters, and publicly available software. The study protocol was pre-registered on the Open Science Framework (https://osf.io/ue6vm/).

Results Our search identified 10,249 records, of which 53 were articles describing 57 QBA methods for summary level data. Of the 57 QBA methods, 51 (89%) were explicitly designed for observational studies, 2 (4%) for non-randomized interventional studies, and 4 (7%) for meta-analyses. There were 29 (51%) QBA methods that addressed unmeasured confounding, 20 (35%) misclassification bias, 5 (9%) selection bias, and 3 (5%) multiple biases. 38 (67%) QBA methods were designed to generate bias-adjusted effect estimates and 18 (32%) were designed to describe how bias could explain away observed findings. 22 (39%) articles provided code or online tools to implement the QBA methods.

Conclusion In this systematic review, we identified a total of 57 QBA methods for summary level epidemiologic data published in the peer-reviewed literature. Future investigators can use this systematic review to identify different QBA methods for summary level epidemiologic data.

Key findings This systematic review identified 57 quantitative bias analysis (QBA) methods for summary level data from observational and non-randomized interventional studies.

Overall, there were 29 QBA methods that addressed unmeasured confounding, 20 that addressed misclassification bias, 5 that addressed selection bias, and 3 that addressed multiple biases.

What this adds to what is known related to methods research within the field of clinical epidemiology?This systematic review provides an overview of the range and characteristics of QBA methods for summary level epidemiologic that are published in the peer-reviewed literature and that can be used by researchers within the field of clinical epidemiology.

What is the implication, what should change now?This systematic review may help future investigators identify different QBA methods for summary level data. However, investigators should carefully review the original manuscripts to ensure that any assumptions are fulfilled, that the necessary bias parameters are available and accurate, and that all interpretations and conclusions are made with caution.

Competing Interest Statement

In the past 36 months, TLL served as a member of the Amgen Methods Advisory Council, for which he received consulting fees and travel support. Dr Ross reported receiving grants from the US Food and Drug Administration; Johnson and Johnson; Medical Device Innovation Consortium; Agency for Healthcare Research and Quality; National Heart, Lung, and Blood Institute; and Arnold Ventures outside the submitted work. Dr Ross also is an expert witness at the request of relator attorneys, the Greene Law Firm, in a qui tam suit alleging violations of the False Claims Act and Anti-Kickback Statute against Biogen Inc. that was settled in September 2022. Dr. Jeffery reported receiving grants from the US Food and Drug Administration; Agency for Healthcare Research and Quality; National Heart, Lung, and Blood Institute; National Center for Advancing Translational Sciences; Nation Institute on Drug Abuse; and American Cancer Society. Dr. Wallach is supported by Arnold Ventures, Johnson & Johnson through the Yale Open Data Access project, and the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health under award 1K01AA028258. Dr. Wallach previously served as a consultant to Hagens Berman Sobol Shapiro LLP and Dugan Law Firm APLC.

Clinical Protocols

https://osf.io/ue6vm/

Funding Statement

This work was supported by the Food and Drug Administration (FDA) of the U.S. Department of Health and Human Services (HHS) as part of a financial assistance award [U01FD005938] totaling $250,000 with 100 percent funded by FDA/HHS. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by FDA/HHS, or the U.S. Government

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Data Availability

All data produced in the present work are contained in the manuscript.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC 4.0 International license.
Back to top
PreviousNext
Posted April 23, 2024.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Quantitative bias analysis methods for summary level epidemiologic data in the peer-reviewed literature: a systematic review
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Quantitative bias analysis methods for summary level epidemiologic data in the peer-reviewed literature: a systematic review
Xiaoting Shi, Ziang Liu, Mingfeng Zhang, Wei Hua, Jie Li, Joo-Yeon Lee, Sai Dharmarajan, Kate Nyhan, Ashley Naimi, Timothy L. Lash, Molly M. Jeffery, Joseph S. Ross, Zeyan Liew, Joshua D. Wallach
medRxiv 2024.04.23.24306205; doi: https://doi.org/10.1101/2024.04.23.24306205
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Quantitative bias analysis methods for summary level epidemiologic data in the peer-reviewed literature: a systematic review
Xiaoting Shi, Ziang Liu, Mingfeng Zhang, Wei Hua, Jie Li, Joo-Yeon Lee, Sai Dharmarajan, Kate Nyhan, Ashley Naimi, Timothy L. Lash, Molly M. Jeffery, Joseph S. Ross, Zeyan Liew, Joshua D. Wallach
medRxiv 2024.04.23.24306205; doi: https://doi.org/10.1101/2024.04.23.24306205

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Epidemiology
Subject Areas
All Articles
  • Addiction Medicine (349)
  • Allergy and Immunology (668)
  • Allergy and Immunology (668)
  • Anesthesia (181)
  • Cardiovascular Medicine (2648)
  • Dentistry and Oral Medicine (316)
  • Dermatology (223)
  • Emergency Medicine (399)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (942)
  • Epidemiology (12228)
  • Forensic Medicine (10)
  • Gastroenterology (759)
  • Genetic and Genomic Medicine (4103)
  • Geriatric Medicine (387)
  • Health Economics (680)
  • Health Informatics (2657)
  • Health Policy (1005)
  • Health Systems and Quality Improvement (985)
  • Hematology (363)
  • HIV/AIDS (851)
  • Infectious Diseases (except HIV/AIDS) (13695)
  • Intensive Care and Critical Care Medicine (797)
  • Medical Education (399)
  • Medical Ethics (109)
  • Nephrology (436)
  • Neurology (3882)
  • Nursing (209)
  • Nutrition (577)
  • Obstetrics and Gynecology (739)
  • Occupational and Environmental Health (695)
  • Oncology (2030)
  • Ophthalmology (585)
  • Orthopedics (240)
  • Otolaryngology (306)
  • Pain Medicine (250)
  • Palliative Medicine (75)
  • Pathology (473)
  • Pediatrics (1115)
  • Pharmacology and Therapeutics (466)
  • Primary Care Research (452)
  • Psychiatry and Clinical Psychology (3432)
  • Public and Global Health (6527)
  • Radiology and Imaging (1403)
  • Rehabilitation Medicine and Physical Therapy (814)
  • Respiratory Medicine (871)
  • Rheumatology (409)
  • Sexual and Reproductive Health (410)
  • Sports Medicine (342)
  • Surgery (448)
  • Toxicology (53)
  • Transplantation (185)
  • Urology (165)