Evaluating Prognostic Bias of Critical Illness Severity Scores Based on Age, Gender, and Primary Language in the USA: A Retrospective Multicenter Study

Evidence before this study We searched PubMed, arXiv, and medRxiv from the inception of the database to July 10, 2022, for articles published without language restrictions. The search terms were (illness severity score OR SOFA OR APACHE-II OR APACHE-IV OR SAPS) AND (evaluation OR performance OR bias) AND ((age OR older OR elderly OR 65 years old OR 80 years old OR subgroup) OR (gender OR Female OR male) OR (language speaking OR English speaking)). Multiple studies have explored the performance among their concerned subgroups with limited patients and hospitals such as over 80, older with sepsis, and surgical patients. Although a small number of studies have presented the performance of scores by age groups, they have not systematically examined the differences and bias between younger and older patients in depth. Few articles analyzed the differences between men and women. No study has discussed the evaluation performance between Non-English and English speakers. We identified that no studies have comprehensively reported the potential bias of clinical scores in the assessment of subgroups classified by age, gender, and English-speaking.

Added value of this study To our best knowledge, we are the first to conduct a systematic bias analysis of the SOFA and APACHE-IVa scores to assess in-hospital outcomes across age (16-44, 45-64, 65-79, and 80-), gender (male and female), and English speaking (Yes and No) subgroups using multicenter data from 189 U.S. hospitals and 173,930 patients episodes. The assessment was performed covering discrimination (AUROC and AUPRC) and calibration (SMR and Calibration belt plot). We found that the AUROCs between the two scores decreased significantly with age. The illness severity exists underestimation for oldest patients and serious overestimation for youngest patients using SOFA score. Both scores demonstrated slightly better AUROCs for males. For Non-English speaking patients, SOFA showed a large reduction in AUROC and very significant underestimation compared to English speakers. Furthermore, there exists higher observed mortality of older patients, females, and Non-English speakers compared to their respective other subgroups using the same SOFA score.

Implications of all the available evidence The aging of the ICU, especially the extremely rapid growth of patients over 80 years old. They exhibit unique characteristics with more comorbidities, frailty, worse prognosis, and the need for more humanistic care, which has evolved into a serious challenge for early clinical triage, diagnosis, and treatment. Females are more likely to withhold pain and not be transferred to the ICU for treatment, which leads to potentially more critical severity illnesses admitted to ICU compared to males. SOFA and APACHE-IVa scores are very important basis and standards for early ICU assessment of illness severity and decision-making. While these general phenomena were noticed in clinical practice of the mentioned subgroups, there is a lack of clear and detailed quantitative analysis of the bias in the use of these scores to protect these vulnerable populations and prevent potential unintentional harm to them. The U.S. is a multicultural and racially integrated country, and the number of Non-English speakers is rising every year which reflects greater socioeconomic and ethnic disparities. Limited communication can also have an impact on patient assessment and treatment. However, the use of the SOFA score for the evaluation of this group of patients has not been reported to date. In this study, we used multicenter data with a large sample size to identify potential bias using the SOFA and APACHE-IVa scores for all mentioned special groups of patients.