ABSTRACT
Missing data are frequently encountered in registries that are used to compare performance across hospitals. The most appropriate method for handling missing data when analysing differences in outcomes between hospitals is unclear. We aimed to compare methods for handling missing data when comparing hospitals on ordinal and dichotomous outcomes. We performed a simulation study using data came from the MR CLEAN registry, a prospective cohort study in 17 hospitals performing endovascular therapy for ischemic stroke in the Netherlands. The investigated methods for handling missing data, both case-mix adjustment variables and outcomes, were complete case analysis (CCA), single imputation, multiple imputation, single imputation with deletion of imputed outcomes and multiple imputation with deletion of imputed outcomes. Data were generated as missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) in three scenarios: (1) 10% missing data in case-mix and outcome; (2) 40% missing data in case-mix and outcome; and (3) 40% missing data in case-mix and outcome with varying degree of missing data among hospitals. Validity and reliability of the methods were compared on the mean squared error (MSE, a summary measure combining bias and precision) relative to the centre effect estimates from the complete reference dataset. For both the ordinal outcome (i.e. the modified Rankin scale) and a common dichotomized version thereof, the MSE of all methods was on average lowest under MCAR and with fewer missing data, and highest with more missing data and under MNAR. The ‘multiple imputation, then deletion’ method had the lowest MSE for both outcomes under all simulated patterns of missing data. Thus, when estimating centre effects on ordinal and dichotomous outcomes in the presence of missing data, the least biased and most precise method to handle these missing data is ‘multiple imputation, then deletion’.
Competing Interest Statement
WvZ has received consulting fees from Codman and Stryker DD has received research grants from Stryker and Bracco Imaging The other authors have nothing to disclose.
Funding Statement
The MR CLEAN Registry was partly funded by TWIN Foundation, Erasmus MC University Medical Centre, Maastricht University Medical Centre, and Amsterdam UMC.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The ethics committee of the Erasmus University MC, Rotterdam, The Netherlands (MEC-2014-235) gave ethical approval for the MR CLEAN registry.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
Data may be obtained from a third party and are not publicly available. The data of the study cannot be made available to other researchers, as Dutch law prohibits data sharing when no patient approval was obtained for sharing coded data. However, syntax or output files of the statistical analyses may be made available for academic purposes upon reasonable request.
