Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Development of a Clinical Decision Support System for Severity Risk Prediction and Triage of COVID-19 Patients at Hospital Admission: an International Multicenter Study

Guangyao Wu, Pei Yang, Henry C. Woodruff, Xiangang Rao, Julien Guiot, Anne-Noelle Frix, Michel Moutschen, Renaud Louis, Jiawei Li, Jing Li, Chenggong Yan, Dan Du, Shengchao Zhao, Yi Ding, Bin Liu, Wenwu Sun, Fabrizio Albarello, Alessandra D’Abramo, Vincenzo Schininà, Emanuele Nicastri, Mariaelena Occhipinti, Giovanni Barisione, Emanuela Barisione, Iva Halilaj, Yuanliang Xie, Xiang Wang, Pierre Lovinfosse, Jianlin Wu, Philippe Lambin
doi: https://doi.org/10.1101/2020.05.01.20053413
Guangyao Wu
2The D-Lab: Department of Precision Medicine, GROW - School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
16Department of Radiology, Affiliated Zhongshan Hospital of Dalian University, Dalian, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: drxieyuanliang{at}163.com g.wu{at}maastrichtuniversity.nl drwangxiang{at}163.com
Pei Yang
1Department of Radiology, The Central Hospital of Wuhan, Huazhong University of Science and Technology, Wuhan, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Henry C. Woodruff
2The D-Lab: Department of Precision Medicine, GROW - School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
3Department of Radiology and Nuclear Medicine, GROW-School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xiangang Rao
8Department of Ultrasound, The Central Hospital of Huangshi, Huangshi, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Julien Guiot
7Department of Respiratory, CHU of Liège, Liège, Belgium
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anne-Noelle Frix
7Department of Respiratory, CHU of Liège, Liège, Belgium
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michel Moutschen
6Department of Infectiology, CHU of Liège, Liège, Belgium
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Renaud Louis
7Department of Respiratory, CHU of Liège, Liège, Belgium
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jiawei Li
9Department of Radiology, China Resources Wuhan Iron and Steel Hospital, Wuhan, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jing Li
10Department of Radiology, The Central Hospital of Shaoyang, Shaoyang, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Chenggong Yan
2The D-Lab: Department of Precision Medicine, GROW - School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
11Department of Medical Imaging Center, Nanfang Hospital, Southern Medical University, Guangzhou, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Dan Du
1Department of Radiology, The Central Hospital of Wuhan, Huazhong University of Science and Technology, Wuhan, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Shengchao Zhao
1Department of Radiology, The Central Hospital of Wuhan, Huazhong University of Science and Technology, Wuhan, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yi Ding
1Department of Radiology, The Central Hospital of Wuhan, Huazhong University of Science and Technology, Wuhan, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bin Liu
1Department of Radiology, The Central Hospital of Wuhan, Huazhong University of Science and Technology, Wuhan, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Wenwu Sun
4Department of Pulmonary and Critical Care Medicine, The Central Hospital of Wuhan, Huazhong University of Science and Technology, Wuhan, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Fabrizio Albarello
12National Institute for Infectious Diseases – IRCCS, Lazzaro Spallanzani, Via Portuense, Rome, Italy
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alessandra D’Abramo
12National Institute for Infectious Diseases – IRCCS, Lazzaro Spallanzani, Via Portuense, Rome, Italy
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Vincenzo Schininà
12National Institute for Infectious Diseases – IRCCS, Lazzaro Spallanzani, Via Portuense, Rome, Italy
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emanuele Nicastri
12National Institute for Infectious Diseases – IRCCS, Lazzaro Spallanzani, Via Portuense, Rome, Italy
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mariaelena Occhipinti
13Department of Biomedical, Clinical and Experimental Sciences “Mario Serio”, University of Florence, Florence, Italy
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Giovanni Barisione
14Department of Respiratory Physiopathology, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emanuela Barisione
15Department of Interventional Pulmonology, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Iva Halilaj
2The D-Lab: Department of Precision Medicine, GROW - School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
MSc
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yuanliang Xie
1Department of Radiology, The Central Hospital of Wuhan, Huazhong University of Science and Technology, Wuhan, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: drxieyuanliang{at}163.com g.wu{at}maastrichtuniversity.nl drwangxiang{at}163.com
Xiang Wang
1Department of Radiology, The Central Hospital of Wuhan, Huazhong University of Science and Technology, Wuhan, China
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: drxieyuanliang{at}163.com g.wu{at}maastrichtuniversity.nl drwangxiang{at}163.com
Pierre Lovinfosse
5Nuclear Medicine and Oncological Imaging, Department of Medical Physics, CHU of Liège, Liège, Belgium
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jianlin Wu
16Department of Radiology, Affiliated Zhongshan Hospital of Dalian University, Dalian, China
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Philippe Lambin
2The D-Lab: Department of Precision Medicine, GROW - School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
3Department of Radiology and Nuclear Medicine, GROW-School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Key points

Question How do nomograms and machine-learning algorithms of severity risk prediction and triage of COVID-19 patients at hospital admission perform?

Findings This model was prospectively validated on six test datasets comprising of 426 patients and yielded AUCs ranging from 0.816 to 0.976, accuracies ranging from 70.8% to 93.8%, sensitivities ranging from 83.7% to 100%, and specificities ranging from 41.0% to 95.7%. The cut-off probability values for low, medium, and high-risk groups were 0.072 and 0.244.

Meaning The findings of this study suggest that our models performs well for the diagnosis and prediction of progression to severe or critical illness of COVID-19 patients and could be used for triage of COVID-19 patients at hospital admission.

IMPORTANCE The outbreak of the coronavirus disease 2019 (COVID-19) has globally strained medical resources and caused significant mortality for severely and critically ill patients. However, the availability of validated nomograms and the machine-learning model to predict severity risk and triage of affected patients is limited.

OBJECTIVE To develop and validate nomograms and machine-learning models for severity risk assessment and triage for COVID-19 patients at hospital admission.

DESIGN, SETTING, AND PARTICIPANTS A retrospective cohort of 299 consecutively hospitalized COVID-19 patients at The Central Hospital of Wuhan, China, from December 23, 2019, to February 13, 2020, was used to train and validate the models. Six cohorts with 426 patients from eight centers in China, Italy, and Belgium, from February 20, 2020, to March 21, 2020, were used to prospectively validate the models.

MAIN OUTCOME AND MEASURES The main outcome was the onset of severe or critical illness during hospitalization. Model performances were quantified using the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity.

RESULTS Of the 299 hospitalized COVID-19 patients in the retrospective cohort, the median age was 50 years ((interquartile range, 35.5-63.0; range, 20–94 years) and 137 (45.8%) were men. Of the 426 hospitalized COVID-19 patients in the prospective cohorts, the median age was 62.0 years ((interquartile range, 50.0-72.0; range, 19-94 years) and 236 (55.4%) were men. The model was prospectively validated on six cohorts yielding AUCs ranging from 0.816 to 0.976, with accuracies ranging from 70.8% to 93.8%, sensitivities ranging from 83.7% to 100%, and specificities ranging from 41.0% to 95.7%. The cut-off values of the low, medium, and high-risk probabilities were 0.072 and 0.244. The developed online calculators can be found at https://covid19risk.ai/.

CONCLUSION AND RELEVANCE The machine learning models, nomograms, and online calculators might be useful for the prediction of onset of severe and critical illness among COVID-19 patients and triage at hospital admission. Further prospective research and clinical feedback are necessary to evaluate the clinical usefulness of this model and to determine whether these models can help optimize medical resources and reduce mortality rates compared with current clinical practices.

Competing Interest Statement

Dr Philippe Lambin reports, within the submitted work, minority shares in The Medical Cloud Company and outside the submitted work grants/sponsored research agreements from Varian medical, Oncoradiomics, ptTheragnostic/DNAmito, Health Innovation Ventures. He received an advisor/presenter fee and/or reimbursement of travel costs/external grant writing fee and/or in kind manpower contribution from Oncoradiomics, BHV, Varian, Elekta, ptTheragnostic and Convert pharmaceuticals. Dr. P. Lambin has shares in the company Oncoradiomics SA, Convert pharmaceuticals SA and and is co-inventor of two issued patents with royalties on radiomics (PCT/NL2014/050248, PCT/NL2014/050728) licensed to Oncoradiomics and one issue patent on mtDNA (PCT/EP2014/059089) licensed to ptTheragnostic/DNAmito, three non-patented invention (softwares) licensed to ptTheragnostic/DNAmito, Oncoradiomics and Health Innovation Ventures and three non-issues, non licensed patents on Deep Learning-Radiomics and LSRT (N2024482, N2024889, N2024889). Dr Henry C. Woodruff has (minority) shares in the company Oncoradiomics. Dr Mariaelena Occhipinti reports grants from Menarini Foundation and Novartis, outside the submitted work. The other authors declare no competing interests.

Clinical Trial

This is a cohort study

Funding Statement

This work was supported from ERC advanced grant (ERC-ADG-2015, n° 694812 - Hypoximmuno), European Program H2020 (ImmunoSABR - n° 733008, PREDICT - ITN - n° 766276, CHAIMELEON - n° 952172, EuCanImage - n° 952103), TRANSCAN Joint Transnational Call 2016 (JTC2016 “CLEARLY” - n° UM 2017-8295), China Scholarships Council (n° 201808210318), and Interreg V-A Euregio Meuse-Rhine (“Euradiomics” - n° EMR4).

Author Declarations

All relevant ethical guidelines have been followed; any necessary IRB and/or ethics committee approvals have been obtained and details of the IRB/oversight body are included in the manuscript.

Yes

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

After the publication of study findings, the data will be available for others to request

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission.
Back to top
PreviousNext
Posted May 07, 2020.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Development of a Clinical Decision Support System for Severity Risk Prediction and Triage of COVID-19 Patients at Hospital Admission: an International Multicenter Study
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Development of a Clinical Decision Support System for Severity Risk Prediction and Triage of COVID-19 Patients at Hospital Admission: an International Multicenter Study
Guangyao Wu, Pei Yang, Henry C. Woodruff, Xiangang Rao, Julien Guiot, Anne-Noelle Frix, Michel Moutschen, Renaud Louis, Jiawei Li, Jing Li, Chenggong Yan, Dan Du, Shengchao Zhao, Yi Ding, Bin Liu, Wenwu Sun, Fabrizio Albarello, Alessandra D’Abramo, Vincenzo Schininà, Emanuele Nicastri, Mariaelena Occhipinti, Giovanni Barisione, Emanuela Barisione, Iva Halilaj, Yuanliang Xie, Xiang Wang, Pierre Lovinfosse, Jianlin Wu, Philippe Lambin
medRxiv 2020.05.01.20053413; doi: https://doi.org/10.1101/2020.05.01.20053413
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Development of a Clinical Decision Support System for Severity Risk Prediction and Triage of COVID-19 Patients at Hospital Admission: an International Multicenter Study
Guangyao Wu, Pei Yang, Henry C. Woodruff, Xiangang Rao, Julien Guiot, Anne-Noelle Frix, Michel Moutschen, Renaud Louis, Jiawei Li, Jing Li, Chenggong Yan, Dan Du, Shengchao Zhao, Yi Ding, Bin Liu, Wenwu Sun, Fabrizio Albarello, Alessandra D’Abramo, Vincenzo Schininà, Emanuele Nicastri, Mariaelena Occhipinti, Giovanni Barisione, Emanuela Barisione, Iva Halilaj, Yuanliang Xie, Xiang Wang, Pierre Lovinfosse, Jianlin Wu, Philippe Lambin
medRxiv 2020.05.01.20053413; doi: https://doi.org/10.1101/2020.05.01.20053413

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Infectious Diseases (except HIV/AIDS)
Subject Areas
All Articles
  • Addiction Medicine (349)
  • Allergy and Immunology (668)
  • Allergy and Immunology (668)
  • Anesthesia (181)
  • Cardiovascular Medicine (2648)
  • Dentistry and Oral Medicine (316)
  • Dermatology (223)
  • Emergency Medicine (399)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (942)
  • Epidemiology (12228)
  • Forensic Medicine (10)
  • Gastroenterology (759)
  • Genetic and Genomic Medicine (4103)
  • Geriatric Medicine (387)
  • Health Economics (680)
  • Health Informatics (2657)
  • Health Policy (1005)
  • Health Systems and Quality Improvement (985)
  • Hematology (363)
  • HIV/AIDS (851)
  • Infectious Diseases (except HIV/AIDS) (13695)
  • Intensive Care and Critical Care Medicine (797)
  • Medical Education (399)
  • Medical Ethics (109)
  • Nephrology (436)
  • Neurology (3882)
  • Nursing (209)
  • Nutrition (577)
  • Obstetrics and Gynecology (739)
  • Occupational and Environmental Health (695)
  • Oncology (2030)
  • Ophthalmology (585)
  • Orthopedics (240)
  • Otolaryngology (306)
  • Pain Medicine (250)
  • Palliative Medicine (75)
  • Pathology (473)
  • Pediatrics (1115)
  • Pharmacology and Therapeutics (466)
  • Primary Care Research (452)
  • Psychiatry and Clinical Psychology (3432)
  • Public and Global Health (6527)
  • Radiology and Imaging (1403)
  • Rehabilitation Medicine and Physical Therapy (814)
  • Respiratory Medicine (871)
  • Rheumatology (409)
  • Sexual and Reproductive Health (410)
  • Sports Medicine (342)
  • Surgery (448)
  • Toxicology (53)
  • Transplantation (185)
  • Urology (165)