Diagnostic Accuracy and Risk Stratification of a Trauma Risk Assessment Tool Among those with Fall Injuries

Study Design and Setting

Using a retrospective cohort design, we pooled trauma registry data between 2017 and 2021 from a single Level-I trauma center that serves a racially diverse community in New York. Data were extracted in June 2022 and the data were analyzed between January and March 2023. This study is among the validation studies focused on exploring the diagnostic accuracies of a novel scoring tool for geriatric trauma patients. We obtained Institutional Review Board (IRB) approval from the New York University Langone Health IRB (i20_01316_MOD05). The authors did not have access to information that could identify individual participants during or after data collection. The results we present follow the reporting guideline of the Standards for Reporting Diagnostic Accuracy.(23)

Inclusion and Exclusion Criteria

We selected adults, 55 years and older, who presented to the ED with traumatic injuries (N=7,634) (Figure 1). We included patients between the ages of 55 and 64 because trauma-related mortality significantly increases after age 55.(24) We excluded patients whose injuries were not related to falls (n=1,102). Duplicate entries (recurrent within-patient ED visits) were identified and we used the most recent encounter (n=740). We performed a listwise deletion for variables with missing entries when the proportion was less than 1% (variable: sex; n=1 (0.0%)). The final analysis included 5,791 adults aged 55 years and older who presented to the ED with fall-related injuries.

• Download figure
• Open in new tab

Figure 1:

Data selection steps

Handling of Missing Data

Three variables had missing values less than 10% - race/ethnicity (n=96, 1.7%), body mass index (n=295 (5.1%)), and Glasgow Coma Scale score (n=539, (9.3%)). We performed multiple imputations after confirming that the missing pattern was missing at random using Little’s test. (25-27) The multiple imputations used the multivariate normal regression with 100 iterations and the imputed value represents the average across the 100 iterations.(28, 29)

Data

From the analytic dataset (n=5,791), we created two separate pools of data: (1) the training dataset (n=4,054) and (2) the test dataset (n=1,737). The training dataset was used for the Receiver Operating Curve (ROC) analysis while the test dataset was used for the internal validation analysis. The split was done in a 70:30 ratio, using simple random sampling with no replacement.

Outcome Variables

The primary outcome measure of interest was in-hospital death. We defined in-hospital death as death occurring either in the ED or during the index admission. In-hospital death was defined as a binary variable (yes or no) and this variable served as the outcome of interest for the ROC and internal validation analyses.

The secondary outcome measures of interest were hospital length of stay and the time to death from fall-related injuries. The hospital length of stay was defined as the duration from hospital admission to final hospital disposition. The final hospital disposition was defined as discharge to home or facility, transfer to another hospital or facility, or death. Hospital length of stay was measured as a continuous variable. Time to death from fall-related injury was defined as the time from admission following the injury to the time of in-hospital death. Patients who were alive at the time of discharge were considered as having right censored events.

Predictor Variable: STTGMA

The predictor variables were the STTGMA risk-triage score and the STTGMA risk categories. The STTGMA risk-triage score was computed using an online Excel calculator.(15) First, we determined if the injury was high or low-energy mechanisms. High-energy falls were falling from heights greater than two meters or collisions with an automobile while low-energy falls were falling from a standing height or lower. Thereafter, we calculated the STTGMA score using the patient’s age, Glasgow Coma Scale score, Charlson Comorbidity Index, and the Abbreviated Injury Severity scores of the head and neck, chest, and extremity and pelvis. Age and Charlson Comorbidity Index were measured as continuous variables. Glasgow Coma Scale score, typically categorized as mild (13 - 15), moderate (8 - 12), and severe (3 to 7) traumatic brain injury was measured as a continuous variable ranging from 3 to 15. Abbreviated Injury Severity, typically categorized as no injury, minor, moderate, serious, severe, critical, or maximal injuries, was scored from 0 to 6, respectively. The STTGMA score is computed as a logarithmic function of these four measures and has a score ranging from 0 to 1. We report the score in percentages, with higher values representing increased injury severity. We generated STTGMA risk categories – minimal (0 – 50%), low (51 – 80%), moderate (81 – 95%), and high (greater than 95%), using the percentile distribution of scores.

Control Variables

We did not control for any additional variables in the ROC and internal validation analyses. For the secondary analysis, we controlled for sex, race/ethnicity, and body mass index. Sex was defined as male or female. Race/ethnicity was defined as non-Hispanic White, non-Hispanic Black, Hispanic, Asian, and other races. Body mass index was defined as normal weight (18.0 – 24.9kg/m₂), underweight (<18.0 kg/m₂), overweight (25.0 – 29.9kg/m₂), and obese (>30.0kg/m₂).

Data Analysis

We reported the frequency distribution and summary statistics in the complete, training, and test datasets and reported the differences across the training and test datasets using the chi-square test, independent sample t-test, and Mann-Whitney U test as appropriate. We computed the median differences in the hospital length of stay and assessed the difference across the predictor and control variables using the Mann-Whitney U test and the Kruskal Wallis test as appropriate. Also, we computed the case fatality rates across the predictor and control variables and assessed differences across the variables using the log-rank test.

For the ROC and internal validation analyses, we first assessed the logarithmic relationship between STTGMA risk-triage scores and in-hospital death and generated the predicted estimates. Thereafter, we created the area under the ROC (AUROC) using the sensitivity on the y-axis and the 1-specificity on the x-axis and reported the accuracy and the 95% confidence interval (CI). AUROC values between 0.7 and 0.8 are interpreted as acceptable, 0.8 to 0.9 as excellent, and higher than 0.9 as outstanding.(30) Also, we computed the Youden index - a measure of a tool’s effectiveness as a diagnostic marker.(23) The Youden index ranges from 0 to 100% and values of 50% or higher are considered adequate.(31)

For the secondary analysis, we performed a quantile regression analysis to assess the association between the STTGMA risk categories and hospital length of stay. We reported the unadjusted and adjusted median differences and the 95% CI. We also performed a time-varying Cox proportional hazard regression analysis to assess the differences between the STTGMA risk categories and the time-to-death from fall-related injuries. We selected a time-varying model since the test of proportionality of strata was significant, evidenced by the crossing of the log-log plot of the STTGMA risk categories, and the significant Schoenfeld residual proportional hazard test.(32) We reported the unadjusted and adjusted hazard risk ratio (mortality risk ratio) and the 95% CI.

Descriptive Characteristics

Among the 5,791 adults who sustained fall injuries, the mean (SD) age was 77.5 (11.4) years (Table 1). The population was predominantly female (63%) and non-Hispanic White (63%). Most of the patients were of either normal weight (39%) or overweight (38%). The injury mechanism was low energy (80%) and 95% of the population had mild traumatic brain injuries. Overall, 45% of the population had no associated comorbidities, and the proportion with minor injuries to the head, chest, and extremities were 67%, 94%, and 52%, respectively. The median (Q1, Q3) STTGMA risk score was 1.3% (0.7%, 2.4%). The median (Q1, Q3) hospital length of stay was 2 (0.0, 5.0) days. There was a total of 122 (2.1%) in-hospital deaths. There were no significant differences in the demographic, injury, and hospital disposition characteristics in the training and test datasets. There were significant differences in the hospital length of stay by sex (p = 0.004), race/ethnicity (p < 0.001), body mass index (p < 0.001), and STTGMA risk categories (p<0.001) (Table 2). Also, there were significant differences in the fall case fatality rates by sex and STTGMA risk categories.

ROC Analysis and Internal Validation

Using the training dataset, a unit increase in STTGMA risk score was associated with an 8% increased odds of fatal fall injury (OR: 1.08; 95% CI: 1.06 - 1.09). STTGMA risk score demonstrated 81% accuracy in predicting in-hospital mortality (95% CI: 75.7 - 86.5) (Figure 2A). The sensitivity and specificity of the STTGMA risk score were 80% and 51%, respectively, and the Youden index was 52%. The internal validation, using the test dataset, showed that a unit increase in STTGMA risk score was associated with a 16% increased odds of death after a fall injury (OR: 1.16; 95% CI: 1.12 - 1.20). STTGMA risk score demonstrated 84% accuracy in predicting in-hospital mortality (95% CI: 75.6 - 92.0) with internal validation (Figure 2B). The sensitivity and specificity of the STTGMA risk scores were 83% and 52%, respectively, and the Youden index was 60%.

• Download figure
• Open in new tab

Figure 2:

Summary of the diagnostic accuracy of the Score for Trauma Triage in the Geriatric and Middle-Aged (STTGMA) in accessing in-hospital mortality among adults 55 years and older with fall injuries

*Youden Index: Ranges from 0 to 100% assuming sensitivity and specificity are of equal value; the higher the value, the better the discriminant property, and values of 50% or higher are acceptable; **AUROC ranging from 0.7 to 0.8 is acceptable, 0.8 to 0.9 is excellent, and values higher than 0.9 as outstanding; AUROC: Area Under Receiver Operating Characteristic

Hospital Length of Stay

We report the unadjusted median change in the hospital length of stay across the demographic characteristics and STTGMA risk categories (Table 3). In the adjusted models, non-Hispanic Black patients (Adjusted Median Difference (aMD): -1.0; 95% CI: -1.5 – -0.5) and Hispanic patients (aMD: -1.0; 95% CI: -1.3 – -0.7) had shorter hospital length of stays compared to those who were non-Hispanic White. Patients who were underweight had longer lengths of stay compared to those who were of normal weight (aMD: 1.0; 95% CI: 0.5 – 1.5). Compared to the minimal risk category, patients in the low (95% CI: 1.8 – 2.2) and moderate (1.7 – 2.3) risk categories had a two-day median increase in their hospital length of stay while those classified as high risk (95% CI: 2.5 – 3.5) had a three-day median increase in their hospital length of stay.

Time to Death

We report the unadjusted mortality risk ratio (hazard risk) across demographic characteristics and STTGMA risk categories (Table 3). In the adjusted models, male patients had two times the mortality risk compared to female patients (adjusted Hazard Ratio (aHR): 2.1; 95% CI: 1.4 – 3.1). Patients classified as having low risk had 2.4 times (95% CI: 1.1 – 5.0) the mortality risk compared to those classified as minimal risk and the mortality risk ratio exponentially increased to 7.4 (95% CI: 3.4 – 16.1) and 33.8 (95% CI: 14.9 – 76.7) in the moderate and high-risk categories.