ABSTRACT
IMPORTANCE Objective and early identification of hospitalized patients, and particularly those with novel coronavirus disease 2019 (COVID-19), who may require mechanical ventilation is of great importance and may aid in delivering timely treatment.
OBJECTIVE To develop, externally validate and prospectively test a transparent deep learning algorithm for predicting 24 hours in advance the need for mechanical ventilation in hospitalized patients and those with COVID-19.
DESIGN Observational cohort study
SETTING Two academic medical centers from January 01, 2016 to December 31, 2019 (Retrospective cohorts) and February 10, 2020 to May 4, 2020 (Prospective cohorts).
PARTICIPANTS Over 31,000 admissions to the intensive care units (ICUs) at two hospitals. Additionally, 777 patients with COVID-19 patients were used for prospective validation. Patients who were placed on mechanical ventilation within four hours of their admission were excluded.
MAIN OUTCOME(S) and MEASURE(S) Electronic health record (EHR) data were extracted on an hourly basis, and a set of 40 features were calculated and passed to an interpretable deep-learning algorithm to predict the future need for mechanical ventilation 24 hours in advance. Additionally, commonly used clinical criteria (based on heart rate, oxygen saturation, respiratory rate, FiO2 and pH) was used to assess future need for mechanical ventilation. Performance of the algorithms were evaluated using the area under receiver-operating characteristic curve (AUC), sensitivity, specificity and positive predictive value.
RESULTS After applying exclusion criteria, the external validation cohort included 3,888 general ICU and 402 COVID-19 patients. The performance of the model (AUC) with a 24-hour prediction horizon at the validation site was 0.882 for the general ICU population and 0.918 for patients with COVID-19. In comparison, commonly used clinical criteria and the ROX score achieved AUCs in the range of 0.773 – 0.782 and 0.768 – 0.810 for the general ICU population and patients with COVID-19, respectively.
CONCLUSIONS and RELEVANCE A generalizable and transparent deep-learning algorithm improves on traditional clinical criteria to predict the need for mechanical ventilation in hospitalized patients, including those with COVID-19. Such an algorithm may help clinicians with optimizing timing of tracheal intubation, better allocation of mechanical ventilation resources and staff, and improve patient care.
Competing Interest Statement
Dr. Nemati is funded by the National Institutes of Health (K01ES025445), Biomedical Advanced Research and Development Authority (HHSO100201900015C), and the Gordon and Betty Moore Foundation (GBMF9052). Dr. Malhotra is a PI on NIH RO1 HL085188, K24 HL132105, T32 HL134632 and co-investigator on R21 HL121794, RO1 HL 119201, RO1 HL081823. ResMed, Inc. provided a philanthropic donation to UC San Diego in support of a sleep center. Dr. Malhotra received funding for medical education from Merck and Livanova. Dr. Mukerji is funded by the National Institutes of Health (K23MH115812) and the Harvard Medical School Elenor and Miles Shore Foundation. Dr. Westover is supported by the Glenn Foundation for Medical Research and the American Federation for Aging Research through a Breakthroughs in Gerontology Grant; the American Academy of Sleep Medicine through an AASM Foundation Strategic Research Award; the Department of Defense through a subcontract from Moberg ICU Solutions, Inc, and by grants from the NIH (1R01NS102190, 1R01NS102574, 1R01NS107291, 1RF1AG064312). Dr. Wardi is supported by the National Foundation of Emergency Medicine and funding from the Gordon and Betty Moore Foundation (GBMF9052). He has received speaker fees from Thermo-Fisher and consulting fees from General Electric. Other co-authors have declared no conflicts of interest and sources of funding.
Funding Statement
Dr. Nemati is funded by the National Institutes of Health (K01ES025445), Biomedical Advanced Research and Development Authority (HHSO100201900015C), and the Gordon and Betty Moore Foundation (GBMF9052). Dr. Malhotra is a PI on NIH RO1 HL085188, K24 HL132105, T32 HL134632 and co-investigator on R21 HL121794, RO1 HL 119201, RO1 HL081823. ResMed, Inc. provided a philanthropic donation to UC San Diego in support of a sleep center. Dr. Malhotra received funding for medical education from Merck and Livanova. Dr. Mukerji is funded by the National Institutes of Health (K23MH115812) and the Harvard Medical School Elenor and Miles Shore Foundation. Dr. Westover is supported by the Glenn Foundation for Medical Research and the American Federation for Aging Research through a Breakthroughs in Gerontology Grant; the American Academy of Sleep Medicine through an AASM Foundation Strategic Research Award; the Department of Defense through a subcontract from Moberg ICU Solutions, Inc, and by grants from the NIH (1R01NS102190, 1R01NS102574, 1R01NS107291, 1RF1AG064312). Dr. Wardi is supported by the National Foundation of Emergency Medicine and funding from the Gordon and Betty Moore Foundation (GBMF9052). He has received speaker fees from Thermo-Fisher and consulting fees from General Electric. Other co-authors have declared no conflicts of interest and sources of funding.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Institutional review board approval of the study was obtained at both sites with a waiver of informed consent (UCSD #191098 and MGH #2013P001024).
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
Access to the computer code used in this research is available upon request to the corresponding author.
