Abstract
Importance Triage quickly identifies critically ill patients, helping to facilitate timely interventions. Many emergency departments use the emergency severity index (ESI) or abnormal vital sign thresholds to identify critically ill patients. However, both rely on fixed thresholds, and false activations detract from efficient care. Prior research suggests that machine-learning approaches may improve triage accuracy, but have relied on information that is often unavailable during the triage process.
Objective We examined whether deep-learning approaches could identify critically ill patients using data immediately available at triage with greater discriminative power than ESI or abnormal vital sign thresholds.
Design Retrospective, cross-sectional study.
Setting An urban tertiary care hospital in the Northeastern United States.
Participants Adult patients presenting to the emergency department from 1/1/2012 - 1/1/2020 were included. Deidentified triage information included structured data (age, sex, initial vital signs, ESI score, and clinical trigger activation due to abnormal vital signs), and textual data (chief complaint) with critical illness (defined as mortality or ICU admission within 24 hours) as the outcome.
Interventions Three progressively complex deep-learning models were trained (logistic regression on structured data, neural network on structured data, and neural network on structured and textual data), and applied to triage information from all patients.
Main Outcomes and Measures The primary outcome was the accuracy of the model for predicting whether patients were critically ill using area under the receiver-operator curve (AUC), as compared to ESI, utilizing a 10-fold cross-validation.
Results 445,925 patients were included, with 60,901 (13.7%) critically ill. Vital sign thresholds identified critically ill patients with AUC 0.521 (95% CI 0.519 -- 0.522), and ESI less than 3 demonstrated AUC 0.672 (95% CI 0.671 -- 0.674), logistic regression classified patients with AUC 0.803 (95% CI 0.802 -- 0.804), neural network with structured data with 0.811 (95% CI 0.807 - 0.815), and the neural network model with textual data with AUC 0.851 (95% CI 0.849 -- 0.852).
Conclusions and Relevance Deep-learning techniques represent a promising method of enhancing the triage process, even when working from limited information. Further research is needed to determine if improved predictions can be translated into meaningful clinical and operational benefits.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was conducted with support from CRICO - Risk Management Foundation of the Harvard Medical Institutions Incorporated (Improving Patient Safety Grant). Role of the Funder: CRICO - Risk Management Foundation of the Harvard Medical Institutions Incorporated, was not involved in the design and conduct of the study; collection, management, analysis, or interpretation of the data; or preparation and review of the manuscript. CRICO did not approve the manuscript and had no input in the decision to submit the manuscript for publication. No publication restrictions apply. The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard University and its affiliated academic healthcare centers.
Author Declarations
All relevant ethical guidelines have been followed; any necessary IRB and/or ethics committee approvals have been obtained and details of the IRB/oversight body are included in the manuscript.
Yes
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
Conflicts of interest: The authors report no conflicts of interest.
Funding/Support: This work was conducted with support from CRICO - Risk Management Foundation of the Harvard Medical Institutions Incorporated (Improving Patient Safety Grant).
Role of the Funder: CRICO - Risk Management Foundation of the Harvard Medical Institutions Incorporated, was not involved in the design and conduct of the study; collection, management, analysis, or interpretation of the data; or preparation and review of the manuscript. CRICO did not approve the manuscript and had no input in the decision to submit the manuscript for publication. No publication restrictions apply. The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard University and its affiliated academic healthcare centers.
Data Availability
The source code of the analysis model is publicly available as a text supplement included with the manuscript, and for download as an online repository. Individual de-identified participant data is available on request in a modified format to accommodate HIPAA SAFE HARBOR criteria.
