Exploring the potential of Claude 2 for risk of bias assessment: Using a large language model to assess randomized controlled trials with RoB 2
View ORCID ProfileAngelika Eisele-Metzger, View ORCID ProfileJudith-Lisa Lieberum, View ORCID ProfileMarkus Toews, View ORCID ProfileWaldemar Siemens, View ORCID ProfileFelix Heilmeyer, Christian Haverkamp, View ORCID ProfileDaniel Boehringer, View ORCID ProfileJoerg J Meerpohl
doi: https://doi.org/10.1101/2024.07.16.24310483
Angelika Eisele-Metzger
1Institute for Evidence in Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg, Germany
2Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany
Judith-Lisa Lieberum
3Eye Center, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg, Germany
Markus Toews
1Institute for Evidence in Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg, Germany
Waldemar Siemens
1Institute for Evidence in Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg, Germany
Felix Heilmeyer
4Institute for Digitalization in Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg, Germany
Christian Haverkamp
4Institute for Digitalization in Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg, Germany
Daniel Boehringer
3Eye Center, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg, Germany
Joerg J Meerpohl
1Institute for Evidence in Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg, Germany
2Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany

DATA AVAILABILITY STATEMENT
Prompt templates, the R code used for analysis, model responses, extracted data and the full sample of Cochrane Reviews assessed for eligibility are stored at open science framework (OSF) and can be accessed via the following link: https://osf.io/2phyt. The source code and documentation for our custom program (Patchbay) are available at https://github.com/daboe01/LLMPatchbay.
Posted July 16, 2024.
Exploring the potential of Claude 2 for risk of bias assessment: Using a large language model to assess randomized controlled trials with RoB 2
Angelika Eisele-Metzger, Judith-Lisa Lieberum, Markus Toews, Waldemar Siemens, Felix Heilmeyer, Christian Haverkamp, Daniel Boehringer, Joerg J Meerpohl
medRxiv 2024.07.16.24310483; doi: https://doi.org/10.1101/2024.07.16.24310483
Exploring the potential of Claude 2 for risk of bias assessment: Using a large language model to assess randomized controlled trials with RoB 2
Angelika Eisele-Metzger, Judith-Lisa Lieberum, Markus Toews, Waldemar Siemens, Felix Heilmeyer, Christian Haverkamp, Daniel Boehringer, Joerg J Meerpohl
medRxiv 2024.07.16.24310483; doi: https://doi.org/10.1101/2024.07.16.24310483
Subject Area
Subject Areas
- Addiction Medicine (349)
- Allergy and Immunology (668)
- Allergy and Immunology (668)
- Anesthesia (181)
- Cardiovascular Medicine (2648)
- Dermatology (223)
- Emergency Medicine (399)
- Epidemiology (12228)
- Forensic Medicine (10)
- Gastroenterology (759)
- Genetic and Genomic Medicine (4103)
- Geriatric Medicine (387)
- Health Economics (680)
- Health Informatics (2657)
- Health Policy (1005)
- Hematology (363)
- HIV/AIDS (851)
- Medical Education (399)
- Medical Ethics (109)
- Nephrology (436)
- Neurology (3882)
- Nursing (209)
- Nutrition (577)
- Oncology (2030)
- Ophthalmology (585)
- Orthopedics (240)
- Otolaryngology (306)
- Pain Medicine (250)
- Palliative Medicine (75)
- Pathology (473)
- Pediatrics (1115)
- Primary Care Research (452)
- Public and Global Health (6527)
- Radiology and Imaging (1403)
- Respiratory Medicine (871)
- Rheumatology (409)
- Sports Medicine (342)
- Surgery (448)
- Toxicology (53)
- Transplantation (185)
- Urology (165)