ABSTRACT
Severity scores assess the acuity of critical illness by penalizing for the deviation of physiologic measurements from normal and aggregating these penalties (also called “weights” or “subscores”) into a final score (or probability) for quantifying the severity of critical illness (or the likelihood of in-hospital mortality). Although these simple additive models are human readable and interpretable, their predictive performance needs to be further improved. To address this need, we argue for replacing these simple additive models with models based on state-of-the-art non-linear supervised learning algorithms (e.g., Random Forest (RF) and eXtreme Gradient Boosting (XGB)). Specifically, we present OASIS+, a variant of the Oxford Acute Severity of Illness Score (OASIS) in which an ensemble of 200 decision trees is used to predict in-hospital mortality based on the 10 same clinical variables in OASIS. Using a test set of 9566 admissions extracted from MIMIC-III database, we show that the performance of OASIS can be substantially improved from AUC score of 0.77 to 0.83 using OASIS+. Moreover, we show that OASIS+ has superior performance compared to eight other commonly used severity scoring methods. Our results underscore the potential of improving existing severity scores by using more sophisticated machine learning algorithms (e.g., ensemble of non-linear decision tress) not just via including additional physiologic measurements.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
YE is supported by a startup funding from Geisinger Health System. The funder had no role in the design of the study, collection, analysis, or interpretation of data or the writing of the manuscript.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The public access to the de-identified MIMIC-III database has been approved by the Beth Israel Deaconess Medical Center (BIDMC) and Massachusetts Institute of Technology (MIT) institutional review boards. All data processing and analyses presented in this study have been conducted in accordance with MIMIC-III guidelines and regulations.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
The datasets presented in the current study are available in the MIMIC III database (https://physionet.org/content/mimiciii/1.4/). Our machine learning model and associated Python scripts are freely available at https://bitbucket.org/i2rlab/oasis/.
