Abstract
Objective We aimed to develop an early warning system for real-time sepsis prediction in the ICU by equipping with interpretation analysis and transfer learning tools to improve the feasibility to deploy the sepsis prediction system, particularly to target cohorts.
Design Retrospective and observational study.
Setting Medical Information Mart for Intensive Care (MIMIC) dataset, the private Historical Database of local Ruijin Hospital (HDRJH), and data collected from Ruijin real-world study.
Patients 6891 patients from MIMIC dataset and 453 patients from HDRJH for model development and 67 cases from Ruijin real-world data for model evaluation.
Interventions None.
Measurements and Main Results Light Gradient Boosting Machine (LightGBM) and multilayer perceptron (MLP) were trained on MIMIC dataset and then finetuned on HDRJH using transfer learning technique. Ultimately, the performance of the sepsis prediction system was further evaluated in the real-world study in the ICU of the target Ruijin Hospital. The area under the receiver operating characteristic curves (AUCs) for LightGBM and MLP models derived from MIMIC were 0.98–0.98 and 0.95–0.96 respectively on MIMIC dataset, and, in comparison, 0.82–0.86 and 0.84–0.87 respectively on HDRJH, from 1–5h preceding. After transfer learning and ensemble learning, the AUCs of the final ensemble model were enhanced to 0.94–0.94 on HDRJH and to 0.86–0.9 in the real-world study in the ICU of the target Ruijin Hospital. In addition, the Shapley additive explanation (SHAP) analysis illustrated the importance of age, antibiotics, net balance, and ventilation for sepsis prediction, making the model interpretable.
Conclusions Our machine learning model allows accurate real-time prediction of sepsis within 5-h preceding. Transfer learning can effectively improve the feasibility to deploy the prediction model in the target cohort, effectively ameliorating the model performance for external validation. SHAP analysis may illuminate the importance of optimizing antibiotic use and restricting fluid management.
Trial registration NCT05088850 (retrospectively registered).
Question We aimed to develop an early warning system for real-time sepsis prediction in the ICU and to improve the feasibility to deploy the system to target cohorts.
Findings Transfer learning technique effectively enhanced the AUCs for LightGBM and MLP models on the target cohort, HDRJH, from 0.82–0.86 and 0.84–0.87 to 0.93-0.94 and 0.92-0.93 for 1-5 hour preceding. Additionally, SHAP analysis illuminated the importance of optimizing antibiotic use and restricting fluid management.
Meaning Transfer learning can improve the feasibility to deploy the prediction model to the target cohort, and SHAP analysis made the prediction model interpretable.
Competing Interest Statement
The authors have declared no competing interest.
Clinical Trial
NCT05088850
Funding Statement
This study was funded by Shanghai Municipal Science and Technology Major Project, the ZHANGJIANG LAB, and the Science and Technology Commission of Shanghai Municipality.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
our study was approved by the Ruijin Hospital Ethics Committee (ethics committee reference number: (2020) Linlunshen No. (140)).
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
Dr. Yaoqing Tang, Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025, China, Tel: +86-021-64370045-611007, Email: tangyaoqing{at}126.com;
Dr. Wenlian Lu, Department of Applied Mathematics, Fudan University, Shanghai, 200433, China, Tel: +86-021-65643250, Email: wenlian{at}fudan.edu.cn
Experiments and elaborations on the transferability and interpretability of the models added; Description of the system architecture removed.
Data Availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
