Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Evaluating Capabilities of Large Language Models: Performance of GPT4 on Surgical Knowledge Assessments

View ORCID ProfileBrendin R Beaulieu-Jones, Sahaj Shah, View ORCID ProfileMargaret T Berrigan, View ORCID ProfileJayson S Marwaha, Shuo-Lun Lai, View ORCID ProfileGabriel A Brat
doi: https://doi.org/10.1101/2023.07.16.23292743
Brendin R Beaulieu-Jones
1Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA
2Department of Biomedical Informatics, Harvard Medical School, Boston, MA
MD MBA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Brendin R Beaulieu-Jones
Sahaj Shah
3Geisinger Commonwealth School of Medicine, Scranton, PA
BS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Margaret T Berrigan
1Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Margaret T Berrigan
Jayson S Marwaha
4Division of Colorectal Surgery, National Taiwan University Hospital, Taipei, Taiwan
MD MBI
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jayson S Marwaha
Shuo-Lun Lai
4Division of Colorectal Surgery, National Taiwan University Hospital, Taipei, Taiwan
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gabriel A Brat
1Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA
2Department of Biomedical Informatics, Harvard Medical School, Boston, MA
MD, FACS, MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Gabriel A Brat
  • For correspondence: gbrat{at}bidmc.harvard.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Article Information

doi 
https://doi.org/10.1101/2023.07.16.23292743
History 
  • July 24, 2023.

Article Versions

  • Version 1 (July 19, 2023 - 08:31).
  • You are viewing Version 2, the most recent version of this article.
Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.

Author Information

  1. Brendin R Beaulieu-Jones, MD MBA1,2,
  2. Sahaj Shah, BS3,
  3. Margaret T Berrigan, MD1,
  4. Jayson S Marwaha, MD MBI4,
  5. Shuo-Lun Lai, MD4 and
  6. Gabriel A Brat, MD, FACS, MPH1,2,*
  1. 1Department of Surgery, Beth Israel Deaconess Medical Center, Boston, MA
  2. 2Department of Biomedical Informatics, Harvard Medical School, Boston, MA
  3. 3Geisinger Commonwealth School of Medicine, Scranton, PA
  4. 4Division of Colorectal Surgery, National Taiwan University Hospital, Taipei, Taiwan
  1. ↵*Corresponding Author: Gabriel A Brat, MD, FACS, MPH, Department of Surgery, Beth Israel Deaconess Medical Center Department of Biomedical Informatics, Harvard Medical School 110 Francis Street, Suite 2G, Boston, MA 02215, gbrat{at}bidmc.harvard.edu
Back to top
PreviousNext
Posted July 24, 2023.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Evaluating Capabilities of Large Language Models: Performance of GPT4 on Surgical Knowledge Assessments
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Evaluating Capabilities of Large Language Models: Performance of GPT4 on Surgical Knowledge Assessments
Brendin R Beaulieu-Jones, Sahaj Shah, Margaret T Berrigan, Jayson S Marwaha, Shuo-Lun Lai, Gabriel A Brat
medRxiv 2023.07.16.23292743; doi: https://doi.org/10.1101/2023.07.16.23292743
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Evaluating Capabilities of Large Language Models: Performance of GPT4 on Surgical Knowledge Assessments
Brendin R Beaulieu-Jones, Sahaj Shah, Margaret T Berrigan, Jayson S Marwaha, Shuo-Lun Lai, Gabriel A Brat
medRxiv 2023.07.16.23292743; doi: https://doi.org/10.1101/2023.07.16.23292743

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Surgery
Subject Areas
All Articles
  • Addiction Medicine (349)
  • Allergy and Immunology (668)
  • Allergy and Immunology (668)
  • Anesthesia (181)
  • Cardiovascular Medicine (2648)
  • Dentistry and Oral Medicine (316)
  • Dermatology (223)
  • Emergency Medicine (399)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (942)
  • Epidemiology (12228)
  • Forensic Medicine (10)
  • Gastroenterology (759)
  • Genetic and Genomic Medicine (4103)
  • Geriatric Medicine (387)
  • Health Economics (680)
  • Health Informatics (2657)
  • Health Policy (1005)
  • Health Systems and Quality Improvement (985)
  • Hematology (363)
  • HIV/AIDS (851)
  • Infectious Diseases (except HIV/AIDS) (13695)
  • Intensive Care and Critical Care Medicine (797)
  • Medical Education (399)
  • Medical Ethics (109)
  • Nephrology (436)
  • Neurology (3882)
  • Nursing (209)
  • Nutrition (577)
  • Obstetrics and Gynecology (739)
  • Occupational and Environmental Health (695)
  • Oncology (2030)
  • Ophthalmology (585)
  • Orthopedics (240)
  • Otolaryngology (306)
  • Pain Medicine (250)
  • Palliative Medicine (75)
  • Pathology (473)
  • Pediatrics (1115)
  • Pharmacology and Therapeutics (466)
  • Primary Care Research (452)
  • Psychiatry and Clinical Psychology (3432)
  • Public and Global Health (6527)
  • Radiology and Imaging (1403)
  • Rehabilitation Medicine and Physical Therapy (814)
  • Respiratory Medicine (871)
  • Rheumatology (409)
  • Sexual and Reproductive Health (410)
  • Sports Medicine (342)
  • Surgery (448)
  • Toxicology (53)
  • Transplantation (185)
  • Urology (165)