Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Reducing diagnostic delays in Acute Hepatic Porphyria using electronic health records data and machine learning: a multicenter development and validation study

View ORCID ProfileBalu Bhasuran, Katharina Schmolly, Yuvraaj Kapoor, Nanditha Lakshmi Jayakumar, Raymond Doan, Jigar Amin, Stephen Meninger, Nathan Cheng, Robert Deering, View ORCID ProfileKarl Anderson, View ORCID ProfileSimon W. Beaven, View ORCID ProfileBruce Wang, View ORCID ProfileVivek A. Rudrapatna
doi: https://doi.org/10.1101/2023.08.30.23293130
Balu Bhasuran
1Bakar Computational Health Sciences Institute, San Francisco, CA, 94143
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Balu Bhasuran
Katharina Schmolly
2David Geffen School of Medicine & Pfleger Liver Institute, University of California Los Angeles, Los Angeles, CA 90095
BS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yuvraaj Kapoor
3Department of Medicine, University of California San Francisco, San Francisco, CA, 94143
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nanditha Lakshmi Jayakumar
3Department of Medicine, University of California San Francisco, San Francisco, CA, 94143
MBBS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Raymond Doan
4Alnylam Pharmaceuticals, Cambridge, Massachusetts, MA 02142
PharmD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jigar Amin
4Alnylam Pharmaceuticals, Cambridge, Massachusetts, MA 02142
PharmD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stephen Meninger
4Alnylam Pharmaceuticals, Cambridge, Massachusetts, MA 02142
PharmD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nathan Cheng
4Alnylam Pharmaceuticals, Cambridge, Massachusetts, MA 02142
PharmD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Robert Deering
4Alnylam Pharmaceuticals, Cambridge, Massachusetts, MA 02142
PharmD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Karl Anderson
5Division of Gastroenterology and Hepatology, University of Texas Medical Branch, School of Medicine, Galveston, TX, 77555
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Karl Anderson
Simon W. Beaven
2David Geffen School of Medicine & Pfleger Liver Institute, University of California Los Angeles, Los Angeles, CA 90095
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Simon W. Beaven
Bruce Wang
3Department of Medicine, University of California San Francisco, San Francisco, CA, 94143
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Bruce Wang
  • For correspondence: bruce.wang{at}ucsf.edu vivek.rudrapatna{at}ucsf.edu
Vivek A. Rudrapatna
1Bakar Computational Health Sciences Institute, San Francisco, CA, 94143
6Division of Gastroenterology and Hepatology, Department of Medicine, University of California, San Francisco, San Francisco, CA, 94143
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Vivek A. Rudrapatna
  • For correspondence: bruce.wang{at}ucsf.edu vivek.rudrapatna{at}ucsf.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

ABSTRACT

Importance Acute Hepatic Porphyria (AHP) is a group of rare but treatable conditions associated with diagnostic delays of fifteen years on average. The advent of electronic health records (EHR) data and machine learning (ML) may improve the timely recognition of rare diseases like AHP. However, prediction models can be difficult to train given the limited case numbers, unstructured EHR data, and selection biases intrinsic to healthcare delivery.

Objective To train and characterize models for identifying patients with AHP.

Design, Setting, and Participants This diagnostic study used structured and notes-based EHR data from two centers at the University of California, UCSF (2012-2022) and UCLA (2019-2022). The data were split into two cohorts (referral, diagnosis) and used to develop models that predict: 1) who will be referred for testing of acute porphyria, amongst those who presented with abdominal pain (a cardinal symptom of AHP), and 2) who will test positive, amongst those referred. The referral cohort consisted of 747 patients referred for testing and 99,849 contemporaneous patients who were not. The diagnosis cohort consisted of 72 confirmed AHP cases and 347 patients who tested negative. Cases were female predominant and 6-75 years old at the time of diagnosis.

Candidate models used a range of architectures. Feature selection was semi-automated and incorporated publicly available data from knowledge graphs.

Main Outcomes and Measures F-score on an outcome-stratified test set

Results The best center-specific referral models achieved an F-score of 86-91%. The best diagnosis model achieved an F-score of 92%. To further test our model, we contacted 372 current patients who lack an AHP diagnosis but were predicted by our models as potentially having it (≥ 10% probability of referral, ≥ 50% of testing positive). However, we were only able to recruit 10 of these patients for biochemical testing, all of whom were negative. Nonetheless, post hoc evaluations suggested that these models could identify 71% of cases earlier than their diagnosis date, saving 1.2 years.

Conclusions and Relevance ML can reduce diagnostic delays in AHP and other rare diseases. Robust recruitment strategies and multicenter coordination will be needed to validate these models before they can be deployed.

Question Can machine learning help identify undiagnosed patients with Acute Hepatic Porphyria (AHP), a group of rare diseases?

Findings Using electronic health records (EHR) data from two centers we developed models to predict: 1) who will be referred for AHP testing, and 2) who will test positive. The best models achieved 89-93% accuracy on the test set. These models appeared capable of recognizing 71% of the cases earlier than their true diagnosis date, reducing diagnostic delays by an average of 1.2 years.

Meaning Machine learning models trained using EHR data can help reduce diagnostic delays in rare diseases like AHP.

Competing Interest Statement

VAR receives research support from Alnylam, Takeda, Merck, Genentech, Blueprint Medicines, Stryker, Mitsubishi Tanabe, and Janssen. BW receives research support from Alnylam and Mitsubishi Tanabe, and honoraria for participation in advisory boards from Alnylam, Mitsubishi-Tanabe, and Disc Medicine. RD, JA, SM, NC, and RD are employees of Alnylam Inc. All authors declare that no actual competing interests exist.

Funding Statement

Research reported in this publication was supported by funding from the UCSF Division of Gastroenterology, the UCSF Bakar Computational Health Sciences Institute, and Alnylam Pharmaceuticals, Inc. This publication was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through UCSF-CTSI Grant Number UL1 TR001872, as well as the UCLA Clinical and Translational Science Institute through grant number UL1TR001881. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The study was approved by the institutional review boards at UCSF (20-31754) and UCLA (21-001260).

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Footnotes

  • ↵* Denotes co-first authorship

Data Availability

The raw data used in this manuscript is available from UCSF and UCLA following the execution of a data use agreement. The analytical code is available at https://github.com/rwelab/AHPPrediction.

https://github.com/rwelab/AHPPrediction

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC 4.0 International license.
Back to top
PreviousNext
Posted August 31, 2023.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Reducing diagnostic delays in Acute Hepatic Porphyria using electronic health records data and machine learning: a multicenter development and validation study
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Reducing diagnostic delays in Acute Hepatic Porphyria using electronic health records data and machine learning: a multicenter development and validation study
Balu Bhasuran, Katharina Schmolly, Yuvraaj Kapoor, Nanditha Lakshmi Jayakumar, Raymond Doan, Jigar Amin, Stephen Meninger, Nathan Cheng, Robert Deering, Karl Anderson, Simon W. Beaven, Bruce Wang, Vivek A. Rudrapatna
medRxiv 2023.08.30.23293130; doi: https://doi.org/10.1101/2023.08.30.23293130
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Reducing diagnostic delays in Acute Hepatic Porphyria using electronic health records data and machine learning: a multicenter development and validation study
Balu Bhasuran, Katharina Schmolly, Yuvraaj Kapoor, Nanditha Lakshmi Jayakumar, Raymond Doan, Jigar Amin, Stephen Meninger, Nathan Cheng, Robert Deering, Karl Anderson, Simon W. Beaven, Bruce Wang, Vivek A. Rudrapatna
medRxiv 2023.08.30.23293130; doi: https://doi.org/10.1101/2023.08.30.23293130

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Gastroenterology
Subject Areas
All Articles
  • Addiction Medicine (349)
  • Allergy and Immunology (668)
  • Allergy and Immunology (668)
  • Anesthesia (181)
  • Cardiovascular Medicine (2648)
  • Dentistry and Oral Medicine (316)
  • Dermatology (223)
  • Emergency Medicine (399)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (942)
  • Epidemiology (12228)
  • Forensic Medicine (10)
  • Gastroenterology (759)
  • Genetic and Genomic Medicine (4103)
  • Geriatric Medicine (387)
  • Health Economics (680)
  • Health Informatics (2657)
  • Health Policy (1005)
  • Health Systems and Quality Improvement (985)
  • Hematology (363)
  • HIV/AIDS (851)
  • Infectious Diseases (except HIV/AIDS) (13695)
  • Intensive Care and Critical Care Medicine (797)
  • Medical Education (399)
  • Medical Ethics (109)
  • Nephrology (436)
  • Neurology (3882)
  • Nursing (209)
  • Nutrition (577)
  • Obstetrics and Gynecology (739)
  • Occupational and Environmental Health (695)
  • Oncology (2030)
  • Ophthalmology (585)
  • Orthopedics (240)
  • Otolaryngology (306)
  • Pain Medicine (250)
  • Palliative Medicine (75)
  • Pathology (473)
  • Pediatrics (1115)
  • Pharmacology and Therapeutics (466)
  • Primary Care Research (452)
  • Psychiatry and Clinical Psychology (3432)
  • Public and Global Health (6527)
  • Radiology and Imaging (1403)
  • Rehabilitation Medicine and Physical Therapy (814)
  • Respiratory Medicine (871)
  • Rheumatology (409)
  • Sexual and Reproductive Health (410)
  • Sports Medicine (342)
  • Surgery (448)
  • Toxicology (53)
  • Transplantation (185)
  • Urology (165)