Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Interpretation of wide confidence intervals in meta-analytic estimates: Is the ‘Absence of Evidence’ ‘Evidence of Absence’?

Sarah L. Miller, Jordan Tuia, View ORCID ProfileVinay Prasad
doi: https://doi.org/10.1101/2023.07.11.23292513
Sarah L. Miller
1University of California San Francisco, 550 16th St, 2nd Fl, San Francisco, CA 94158
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jordan Tuia
1University of California San Francisco, 550 16th St, 2nd Fl, San Francisco, CA 94158
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Vinay Prasad
1University of California San Francisco, 550 16th St, 2nd Fl, San Francisco, CA 94158
MD, MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Vinay Prasad
  • For correspondence: vinayak.prasad{at}ucsf.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Abstract

Introduction Recently, a Cochrane review by Jefferson et al. on physical interventions to slow the spread of respiratory viruses concluded that, “Wearing masks in the community probably makes little or no difference to the outcome of laboratory-confirmed influenza/SARS-CoV-2 compared to not wearing masks”, though this finding had a wide confidence interval. Cochrane issued a rare clarifying statement, fueling controversy. We sought to contextualize the findings of the review by Jefferson et al.

Methods We searched for consecutive reviews by Cochrane published on or before March 9th, 2023. We included studies where a central finding showed an intervention offered no statistically significant benefit, and ascertained the language used by reviewers to describe that result. We compare this to the report by Jefferson et al., and deemed it consistent or inconsistent with the language of their report.

Results We found between November 21st, 2022, and March 9th, 2023, there were 20 Cochrane reviews that met the inclusion criteria. We found that 95% (n = 19) of the reviews used language that was consistent with Jefferson’s findings, while 5% (n = 1) used language inconsistent with Jefferson’s conclusion, describing the effect of the intervention on the outcome as “unclear”.

Discussion Most reviews performed by Cochrane conclude that interventions which fail to show statistically significant benefits make “no difference” have “no effect” or do not “increase or decrease” the outcome, and this occurs despite wide confidence intervals. The conclusions by Jefferson et al. are consistent with Cochrane reporting guidelines and clarification from the organization was unjustified.

Competing Interest Statement

Disclosure: Vinay Prasad′s Disclosures. (Research funding) Arnold Ventures (Royalties) Johns Hopkins Press, Medscape, and MedPage (Honoraria) Grand Rounds/lectures from universities, medical centers, non–profits, and professional societies. (Consulting) UnitedHealthcare and OptumRX. (Other) Plenary Session podcast has Patreon backers, YouTube, and Substack. All other authors have no financial nor non–financial conflicts of interest to report.

Funding Statement

None. VP is funded by Arnold Ventures to study Low Value Care. This study received no specific funding.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Footnotes

  • Funding: None. VP is funded by Arnold Ventures to study Low Value Care. This study received no specific funding.

  • Disclosure: Vinay Prasad’s Disclosures. (Research funding) Arnold Ventures (Royalties) Johns Hopkins Press, Medscape, and MedPage (Honoraria) Grand Rounds/lectures from universities, medical centers, non-profits, and professional societies. (Consulting) UnitedHealthcare and OptumRX. (Other) Plenary Session podcast has Patreon backers, YouTube, and Substack. All other authors have no financial nor non-financial conflicts of interest to report.

Data Availability

All data produced in the present work are contained in the manuscript

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-ND 4.0 International license.
Back to top
PreviousNext
Posted July 14, 2023.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Interpretation of wide confidence intervals in meta-analytic estimates: Is the ‘Absence of Evidence’ ‘Evidence of Absence’?
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Interpretation of wide confidence intervals in meta-analytic estimates: Is the ‘Absence of Evidence’ ‘Evidence of Absence’?
Sarah L. Miller, Jordan Tuia, Vinay Prasad
medRxiv 2023.07.11.23292513; doi: https://doi.org/10.1101/2023.07.11.23292513
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Interpretation of wide confidence intervals in meta-analytic estimates: Is the ‘Absence of Evidence’ ‘Evidence of Absence’?
Sarah L. Miller, Jordan Tuia, Vinay Prasad
medRxiv 2023.07.11.23292513; doi: https://doi.org/10.1101/2023.07.11.23292513

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Epidemiology
Subject Areas
All Articles
  • Addiction Medicine (349)
  • Allergy and Immunology (668)
  • Allergy and Immunology (668)
  • Anesthesia (181)
  • Cardiovascular Medicine (2648)
  • Dentistry and Oral Medicine (316)
  • Dermatology (223)
  • Emergency Medicine (399)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (942)
  • Epidemiology (12228)
  • Forensic Medicine (10)
  • Gastroenterology (759)
  • Genetic and Genomic Medicine (4103)
  • Geriatric Medicine (387)
  • Health Economics (680)
  • Health Informatics (2657)
  • Health Policy (1005)
  • Health Systems and Quality Improvement (985)
  • Hematology (363)
  • HIV/AIDS (851)
  • Infectious Diseases (except HIV/AIDS) (13695)
  • Intensive Care and Critical Care Medicine (797)
  • Medical Education (399)
  • Medical Ethics (109)
  • Nephrology (436)
  • Neurology (3882)
  • Nursing (209)
  • Nutrition (577)
  • Obstetrics and Gynecology (739)
  • Occupational and Environmental Health (695)
  • Oncology (2030)
  • Ophthalmology (585)
  • Orthopedics (240)
  • Otolaryngology (306)
  • Pain Medicine (250)
  • Palliative Medicine (75)
  • Pathology (473)
  • Pediatrics (1115)
  • Pharmacology and Therapeutics (466)
  • Primary Care Research (452)
  • Psychiatry and Clinical Psychology (3432)
  • Public and Global Health (6527)
  • Radiology and Imaging (1403)
  • Rehabilitation Medicine and Physical Therapy (814)
  • Respiratory Medicine (871)
  • Rheumatology (409)
  • Sexual and Reproductive Health (410)
  • Sports Medicine (342)
  • Surgery (448)
  • Toxicology (53)
  • Transplantation (185)
  • Urology (165)