1. Abstract
Background Treatment of acute ischemic stroke is heavily contingent upon time, as there is a strong relationship between time clock and tissue progression. We sought to a develop a deep learning algorithm for classifying time since stroke (TSS) from MR images by comparison to neuroradiologist assessments of imaging signal mismatch and evaluation on external data.
Methods This retrospective study involved patients who underwent MRI from 2011-2019. Models were trained to classify TSS within 4.5 hours; performance metrics with confidence intervals were reported on both internal and external evaluation sets.
Results A total of 772 patients (66 ± 9 years, 319 women) were used for model development and evaluation. Three board-certified neuroradiologists’ assessments, based on majority vote, yielded a sensitivity of 0.62, a specificity of 0.86, and a Fleiss’ kappa of 0.46. The deep learning method performed similarly to radiologists and outperformed previously reported methods, with the best model achieving an average evaluation accuracy, sensitivity, and specificity of 0.726, 0.712, and 0.741, on an internal cohort and 0.724, 0.757, and 0.679, respectively, on an external, unseen evaluation cohort from another institution.
Conclusion This model achieved higher generalization performance on external evaluation datasets than the current state of the art for TSS classification.
Competing Interest Statement
The authors of this manuscript declare relationships with the following companies: Kambiz Nael, MD, serves as a consultant for Olea Medical. All other authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Funding Statement
This work was supported by the United States National Institutes of Health (NIH) grants R01NS100806 and T32EB016640, and an NVIDIA Academic Hardware Grant.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The Medical Institutional Review Board #3 (MIRB3) of UCLA gave ethical approval for this work under IRB#18-000329, A Machine Learning Approach to Classifying Time Since Stroke using Medical Imaging. Patient records were collected in accordance with IRB approval and HIPAA compliance standards. Informed consent was waived under Exemption 4 for retrospective data.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
Funding: The work in this study was supported by the following funding sources: NIH T32EB016640-07, NIH R01NS100806-02, and NVIDIA Academic Hardware Grant.
Abstract Updated; Methods revised for clarity; Table 1 and Table 4 updated
Data Availability
The datasets presented in this article are not readily available due to protection of patient privacy. We are willing to validate other models internally on our data as part of collaborations. Program code and derived data (e.g., model weights) will be made available upon publication. Requests to access the datasets should be directed to the corresponding author.
