Abstract
Importance Trauma causes over four million deaths annually, predominantly in low- and middle-income countries. Implementing trauma quality improvement programs may improve outcomes, and though extensively used, high-quality evidence of their effectiveness is scarce.
Objective To assess if implementing a trauma quality improvement program using on audit filters improves trauma patients’ outcomes.
Design A controlled interrupted time-series study.
Setting Prospective, multicenter study across four tertiary care hospitals in urban India between 2017-2022.
Participants Adult patients admitted to participating hospitals with history a of trauma, defined as having any of the external causes listed in block V01-Y36, chapter XX of the ICD-10 as reason for admission.
Intervention In the intervention arm (two hospitals), a trauma quality improvement program using audit filters was implemented after a one-year observation period. The control arm (two hospitals) continued baseline data collection without intervention throughout the study period.
Main Outcomes All-cause mortality (in-hospital and at 30 days). For time series analysis, segmented regression with a generalized additive model (GAM) assessed the effect on in-hospital mortality. Secondary analysis using difference-in-differences and linear regression assessed in-hospital and 30-day mortality. The study was not powered for time series analysis on 30-day mortality.
Results We included 10143 patients, median age 35 (IQR 26 – 50), 83% men. Using time series analysis, we observed a significant reduction in in-hospital mortality (32% vs 24%; OR 0.56, 95% CI 0.4-0.77, p<0.001) in the intervention arm, with no significant change in the control arm. Difference-in-differences analysis found a significant reduction in 30-day mortality (39% vs 26%; DiD estimate -0.15 95% CI -0.19 to -0.11, p<0.001) and in-hospital mortality (32% vs 24%; DiD estimate -0.12 95% CI -0.16 to -0.09, p<0.001). However, external factors such as the opening of a dedicated trauma center at one intervention hospital and the COVID-19 pandemic may have influenced these results.
Conclusion Implementing a trauma quality improvement program using audit filters may reduce mortality. More research is needed to confirm these findings across different settings and to understand by which mechanisms these programs mediate the effect and ensure sustainability in terms of improving outcomes.
Trial registration Trauma Audit Filter Trial, ClinicalTrials.gov ID NCT03235388, https://clinicaltrials.gov/study/NCT03235388
Question Does implementing a trauma quality improvement program using audit filters improve mortality in adult trauma patients?
Findings In this prospective, multicentre, controlled interrupted time series including 10143 patients across four tertiary care hospitals in urban India, we observed a significant reduction in all-cause 30-day and in-hospital mortality after implementing a trauma quality improvement program using audit filters.
Meaning implementing trauma quality improvement programs using audit filters may reduce mortality. However, these programs require local adaptation, and their effectiveness is dependent on the setting, context and maturity of the health care system.
Introduction
Trauma causes more than four million deaths per year and is the most common cause of death in individuals aged 10-29 years1. Over two million lives could be saved each year if mortality rates in low- and middle-income countries (LMICs) were the same as those in high-income countries (HICs).2 The establishment of trauma systems, encompassing prevention, pre- and intrahospital care, and rehabilitation, is associated with reduced mortality in HICs.3–6 Trauma quality improvement programs, a core part of trauma systems, involve an ongoing process that includes identifying deviations from care standards, analyzing these findings, evaluating factors for improvement, and implementing corrective action plans to improve patient care.7–9
Audit filters, in some areas referred to as process quality indicators, are predefined statements that represent ideal care standards.8,10 An example of an audit filter is “Patients with a Glasgow Coma Scale (GCS) score of less than 8 should receive a definitive airway.” These filters are used to flag individual patient cases so that the care can be reviewed, potential opportunities for improvement can be identified, and corrective strategies can be implemented. This process is resource intensive because it requires continuous data collection and filter violation tracking.8,11
The American College of Surgeons released the first 22 trauma audit filters as a part of the guidelines on trauma care in 198712. Since then, the use of audit filters has been widely applied in HIC and recommended by the World Health Organization (WHO) as a way to meet the global demand for improved care of individuals with injures, despite the lack of evidence that doing so improves patient outcomes. A 2009 systematic Cochrane review on trauma audit filters revealed no studies of sufficient quality for inclusion.11 Our aim was therefore to assess whether implementing a trauma quality improvement program using audit filters improves trauma patient outcomes.
Methods
Study design and setting
We conducted a controlled interrupted time series study across four hospitals in urban India (ClinicalTrials.gov identifier NCT0323538813). All tertiary care hospitals have approximately 1500 beds and in-house clinical specialties to care for trauma patients. None of the hospitals used any structured trauma quality improvement process. The study had three phases: an observation phase lasting 14 months in all four hospitals to establish baseline outcomes, an implementation phase lasting six months, during which two hospitals were randomized to implement a trauma quality improvement program with audit filters, and an intervention phase lasting 41 months. The intervention phase was extended by 18 months due to the COVID-19 pandemic. We performed an interim analysis 15 months after the initiation of the intervention to asses the quality of the data and identify unexpected changes in outcomes.
Participants
Inclusion criteria and enrollment
We included adults aged 18 years and older admitted for in-hospital care with a history of trauma, defined by International Classification of Diseases, tenth revision (ICD-10) codes V01-Y36 for external causes of morbidity and mortality, as the reason for admission. Project officers were trained to record vital signs with standard equipment for patients across rotating shifts (day, evening, and night). Each shift comprised six hours for enrolling new patients in the emergency department and two hours for follow-up of previously included patients. We also included patients retrospectively due to unexpectedly low prospective inclusion rates, especially during the COVID-19 pandemic. Using hospital records, we identified all trauma patients admitted monthly to each hospital and collected data retrospectively from a random sample of these patients.
Data collection and management
Initial data collection was performed using paper forms, and the data were then periodically entered into a digital data collection tool and uploaded to secure servers. Patient identifiers were not uploaded, and identification was only possible through the original paper records, which were stored at each hospital in compliance with local regulations. Basic data validation was performed at the time of entry, and double entry of all variables was performed after all the data had been collected to minimize the risk of transfer errors. Quality control visits were conducted each quarter with project officers from different hospitals evaluating the data collection processes based on predefined criteria.
Outcomes
Our primary outcome was all-cause mortality at 30 days after admission to the hospital. This information was collected from medical records if the patient died during the hospital stay; otherwise, it was obtained via telephone follow-up. We collected data on secondary outcomes, including in-hospital mortality, length of stay and intensive care unit (ICU) admissions. Data on several covariates, such as vital parameters, injury type and examinations, were collected to allow us to describe and compare cohorts and adjust for case-mix differences and differences between hospitals over time. For the retrospectively included patients, data on the date of admission, age, sex, mechanism of injury, and in-hospital mortality were collected.
Study execution
One principal investigator and one coprincipal investigator were recruited for each participating hospital. These were all experienced trauma clinicians with extensive local knowledge and research interest.
Observation phase (Month 1-14)
The participating hospitals started data collection at the same time to establish baseline outcomes. Weekly meetings were held with all project officers and the core research team to identify and address any issues related to data collection.
Implementation phase (Month 15-20)
The participating hospitals were paired so that the two hospitals with the highest and lowest volumes formed one pair, and the two remaining hospitals formed the second pair. One pair of hospitals was then randomly selected to implement trauma audit filters, becoming intervention hospitals, while the other two were control hospitals. The process of data collection remained the same at all sites. At the intervention hospitals, a two-day session on the background and rationale of trauma audit filters was held by representatives from the core research team. This session was attended by a multidisciplinary team of surgeons/physicians, anesthetists, administrators and nurses involved in trauma care. A local trauma audit review board was established at each intervention hospital.
We conducted an anonymous Delphi survey at each intervention site to select audit filters14. A list of implemented audit filters is included as supplementary material. A second project officer, who prospectively included patients and noted any violations to the defined audit filters by direct observation and chart reviews, was employed at each intervention center. A report based on these data, flagging and presenting cases of violations, was prepared by the core research team for each review meeting. Participants from the core research team attended audit filter review meetings to facilitate the discussion and formulate corrective strategies accordingly.
Intervention phase (Months 21-42)
The intervention hospitals continued the data collection, both for base data and for audit filter deviations. The trauma audit review board continued to analyze deviations, implementing corrective strategies. The meetings were held without participation from the core research team. Three months after the intervention phase started, the largest intervention hospital opened a dedicated trauma center. The control hospitals continued with base data collection.
Statistical analysis
For our primary analysis, we applied a segmented generalized additive model (GAM) to assess the impact of the intervention on outcomes.15 This approach was chosen to allow us to accommodate for nonlinearity in the intervention effect while adjusting for potential autocorrelation, seasonality, and case-mix differences. We adjusted for month using a cubic spline with 12 knots to allow for seasonality. The data were pooled from the intervention and control centers by month, with each observation representing averaged data for all patients during that month. We investigated the occurrence of autocorrelation by the Ljung-Box test. We developed a counterfactual model to compare predictions with or without the intervention.
We performed a secondary analysis with patient-level data for formal comparisons between the control and intervention arms using a linear regression difference-in-differences analysis while adjusting for confounders. We used R for the statistical analyses16, adopting a 95% confidence level and 5% significance level. A GAM was applied using the mgcv R package. The Holm procedure was applied to adjust for multiple tests.17 Unadjusted pre/post analyses were performed using two-sample Z tests for proportions.
Subgroup and sensitivity analyses
We conducted two prespecified subgroup analyses for major trauma and potentially salvageable trauma patients. We defined a major trauma patient as any patient who was in the hospital for more than three days with an injury severity score (ISS) >15 or who died within three days of arrival and a potentially salvageable trauma patient as any patient with 15<ISS<24. We conducted three sensitivity analyses: an analysis excluding the implementation phase, an analysis excluding the period after a trauma center opened at one intervention hospital and an analysis excluding the COVID-19 phase from the time-series model.
Sample size considerations
The sample size requirements for interrupted time series analyses depend on several factors, including model complexity, data variance and the temporal spread of observations. For our primary analysis, we chose to adhere to published guidelines stating that at least twelve observations are needed during the observation phase and twelve observations are needed during the intervention phase and that each observation should be an aggregate of at least 100 patients.18 Our use of a large GAM introduced extra complexity, and we estimated that 100 patients/arm/month and an extended intervention phase were needed to detect the potential impact of the intervention. This sample size allowed us to detect a reduction in mortality from 20% to 15% (power 0.8, alpha 0.05) using a pre-post design.
Missing data and data processing
We analyzed the associations between missing data for the included covariates and outcomes using ANOVA-F test for continuous variables and the chi-squared test for categorical variables. If a significant correlation was found, we conducted further investigation to determine the type of missingness. Only patients with complete data were included in the statistical analysis. Analysis for missing data was performed using the R package finalfit.19 We calculated the Abbreviated Injury Scale (AIS) and the subsequent ISS based on ICD-10 codes using the R package icdpicr.20
Ethical considerations
The need for informed consent to participate in the intervention was not deemed applicable, and we were granted waivers of informed consent for recording vital signs, demographic parameters and in-hospital outcomes. The project officers obtained written consent for telephone follow-up. Ethical approval was granted by the Swedish Ethical Review Authority (approved 2017-06-07 2017/930-31/2), as well as by all local ethical review boards at each participating hospital (Maulana Azad Medical College (MAMC) - approved 2017-07-19 F.1/IEC/MAMC/(57/02/2017/No 113. SSKM/IPGME&R, Kolkata – approved 2017-08-21, IPGME&R/IEC/2017/396. JJ Hospital, Mumbai – approved 2017-08-22, No. IEC/Pharm/CT/111/A/2017. St. Johns, Bangalore – approved 2017-08-24, 160/2017).
Results
We included 10143 patients between October 2017 and October 2022. Of these patients, 4126 were prospectively included (Figure 1). We included an average of 166 standard deviation (SD) (61) patients per month, 45 out of 61 months did not reach the target number of inclusions, due to low inclusion rates and lower number of patients during COVID-19. The included patients, key characteristics and number of missing data are listed in Table 1 for all patients and Table 2 for prospectively included patients. We found no significant association between missing data and the predictors used, except for one month in the control arm, which is unlikely to influence the results of the time series analysis. In the intervention arm, we prospectively screened 1454 patients for audit filter violations. A complete list of audit filters, filters with the most violations and additional details on missing data is available in the supplementary material.
In the intervention arm, the intervention phase was associated with an absolute 14% reduction (95% CI -18% to -9.6%, p<0.001) in the 30-day mortality rate in the prospectively included cohort, and an 8% reduction (95% CI -11% to -5.3%, p<0.001) in the in-hospital mortality rate in the total cohort. In the control arm there was no significant change in the 30-day mortality, but a 3% increase in the in-hospital mortality rate (95% CI 0.38% to 5.0%, p=0.032)
Time series analysis for in-hospital mortality
The time series analysis of all 10143 included patients revealed that the intervention phase was associated with a significant reduction in the odds of in-hospital mortality in the intervention arm (odds ratio (OR): 0.56, 95% CI 0.4-0.77, p<0.001 while no significant change was detected in the control arm, (OR: 0.94, 95% CI 0.5-1.75, p=0.83) (Figure 2).The parameter month was not significant (p=0.48), indicating no significant seasonality. The Ljung-Box test of residuals did not return any significant results, indicating no significant autocorrelation.
Difference in differences analysis
Formally comparing the control and intervention arms in the difference-in-differences analysis, including 3317 prospectively included patients, we found that the intervention was associated with a 15% reduction in the 30-day mortality rate (-0.15, 95% CI -0.19 to -0.11, p<0.001) and an 12% reduction in the in-hospital mortality rate (-0.12, 95% CI -0.16 to -0.09, p<0.001) in the intervention arm compared to the control arm after adjusting for sex, age, GCS score, Injury Severity Score (ISS) and shock. For major trauma and potentially salvageable trauma patients the reduction in the mortality was more pronounced (Table 3).
Sensitivity analysis
We performed three sensitivity analyses for the time-series analysis: excluding the implementation phase, excluding the time after the trauma center opened and excluding the main COVID-19 period. We defined the exclusion period for COVID-19 as the time when we were not able to include prospective patients, April 2020 and December 2020. All analyses showed a significant reduction in the in-hospital mortality rate in the intervention arm. One analysis, excluding the implementation phase, revealed in a significant increase in the in-hospital mortality rate in the control arm, with the remaining analysis showing no significant change in the control arm. Results are available in the supplementary material.
Discussion
Our results indicate that implementing a trauma quality improvement program using audit filters was associated with a substantial, significant reduction in both 30-day and in-hospital mortality rates. For severely injured patients, the effect is more pronounced. We also observed increased use of ultrasound, intubations and admissions to the ICU, reflecting changes in care processes in alignment with the audit filters. However, the largest hospital in the intervention arm opened a trauma center equipped with in-house CT scanner, multiple operating rooms, improved resuscitation capabilities and an increased number of ICU beds nine months after the beginning of the implementation phase. Although our sensitivity analysis revealed a decrease in mortality before the trauma center opened, it remains challenging to separate the impacts of the trauma center and the quality improvement program or determine whether their effects were synergistic.
Several studies have reported improved mortality after trauma quality improvement implementation21–25. However, definitions of trauma quality improvement vary, with heterogeneous interventions reported. Two studies conducted over 20 years ago in Germany and Thailand used trauma audit filters and reported reduced mortality and improved care processes26,27. In more recent years, two studies from Ghana and Cameroon developed local trauma audit filters for implementation28,29. These filters differ substantially from those initially developed by ASCOT and later suggested in the WHO guidelines. In our previous study, the usefulness of the filters developed in Ghana and Cameroon was deemed high in India, compared filters developed in HICs.14
In HIC trauma quality improvement programs and audit filters have been criticized for being inefficient in detecting opportunities for improvement and being costly.30 In obstetric care, audit processes have shown to improve outcomes while also emphasizing the importance of adaption31, highlighting that standards of care need to be developed considering local priorities and knowledge to ensure that improvement efforts are directed to areas in greatest need. This is especially important in settings with less developed trauma systems.
The maturity level of a trauma system is linked to reduced mortality and preventable deaths.5,32 Therefore, the effectiveness of these programs in reducing mortality likely varies with system maturity. In a similar urban Indian setting, over 50% of trauma deaths where estimated to be preventable.33 Our results can likely be generalized to settings with similarly mature trauma systems, tough local adaptations and contextual factors greatly influence their success. The core process of data collection, case revision and multidisciplinary review can be utilized to empower providers to identify areas for improvement, learning, education to develop of the care provided. This can be conducted for trauma systems at all levels, even though data quality is a known barrier to this in LMICs.24,34,35
Our study has several limitations. First, the introduction of a new trauma center likely influenced patient outcomes, posing a significant confounder. Second, lower than expected inclusion rates required the collection of retrospective data, preventing the main time-series analysis of 30-day mortality. However, the study was well powered for the pre-post design, allowing us to estimate the effect of the program on this outcome. Third, an unexpected difference in mortality between the experimental arms required more power to detect changes in the control arms. Using a difference-in-differences approach, we relied on trends in the control centers rather than mortality levels. Fourth, the COVID-19 pandemic significantly impacted the participating centers. One intervention center was converted to a COVID-19 hospital, reducing trauma patient intake. Quarantine restrictions also decreased the overall number of trauma patients. Despite extending the study by 18 months and adjusting for the pandemic’s effects in the sensitivity analysis, its long-term impacts were likely beyond our ability to fully account for. The pandemic led to the suspension of review meetings, necessitating a restart postpandemic. Last, both intervention hospitals receive a high number of transferred patients at varying intervals after the time of injury, potentially introducing bias, as patients may have died before arriving at the study hospital.
Despite these limitations, this is the first quasi-experimental study attempting to assess the impact of quality improvement programs using audit filters on patient outcomes. We conducted a broad and comprehensive statistical analysis that supports our main findings, which are also in alignment with previous research. We believe that there is knowledge on how to select and adapt audit filters and implement a review process, but our understanding of how this translates into changes in care and outcomes is lacking. We also need better understanding of the implementation and sustainability of these programs in complex health care systems, including how to best tailor them to local needs and detect potential negative effects, especially for other patient populations also competing for care resources.
In conclusion, our results suggest that system-level changes and data-driven quality improvement programs using audit filters may reduce mortality in trauma patients. However, the challenge to understand what makes these programs effective, useful and sustainable in terms of improving outcomes remains.
Data Availability
All data in the present study consists of sensitive patient information. To protect the privacy of the participants, this data cannot be released outside the scope of this research collaboration in its current form. We are in the process of creating an anonymised version of this data, which we aim to release openly. This anonymized dataset will be published alongside a data descriptor paper to facilitate open-access use.
Author contributions
MGW conceived the study and has been the PI. MGW, KS, MK, NR, SD, LFT, and MP contributed to the design of the study. SD was the project manager during study execution. MGW, KS, MK, NR, SD, JB, MJ, SR, MLN, RS, AM, SC, GB, DB, TK contributed to the execution of the study. JB prepared the data and conducted the statistical analysis. MGW and MP reviewed the results of the statistical analysis. JB drafted the first version of the article. All authors contributed to interpretation of data and critical revisions of the work for important intellectual content. MGW is the guarantor.
Funding
This work was supported by the Swedish Research Council (2016-02041).
Competing interests
Non declared.
Acknowledgement
We would like to thank Johan von Schreeb, professor at the department of Global Public Health at the Karolinska Institute and Charles Mock, Professor Emeritus of Surgery and Epidemiology, University of Washington for their valuable input during this entire project. A warm thank you to Manjula Ranagatti and all project officers that have dedicated their time and expertise to this project and ensuring the quality of data collection.
JB and MGW had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Footnotes
This version has been revised: Language revision Minor analysis update to correct status for status at 24 hours and in hospital mortality Addition of study flowchart Updated supplementary material Updated figure 1
References
