RT Journal Article
SR Electronic
T1 A novel workflow for the safe and effective integration of AI as supporting reader in double reading breast cancer screening: A large-scale retrospective evaluation
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 2022.06.22.22276751
DO 10.1101/2022.06.22.22276751
A1 Ng, Annie Y.
A1 Glocker, Ben
A1 Oberije, Cary
A1 Fox, Georgia
A1 Nash, Jonathan
A1 Karpati, Edith
A1 Kerruish, Sarah
A1 Kecskemethy, Peter D.
YR 2022
UL http://medrxiv.org/content/early/2022/06/23/2022.06.22.22276751.abstract
AB Objectives To evaluate the effectiveness of a novel strategy for using AI as a supporting reader for the detection of breast cancer in mammography-based double reading screening practice. Instead of replacing a human reader, here AI serves as the second reader only if it agrees with the recall/no-recall decision of the first human reader. Otherwise, a second human reader makes an assessment, enacting standard human double reading.Design Retrospective large-scale, multi-site, multi-device, evaluation study.Participants 280,594 cases from 180,542 female participants who were screened for breast cancer with digital mammography between 2009 and 2019 at seven screening sites in two countries (UK and Hungary).Main outcome measures Primary outcome measures were cancer detection rate, recall rate, sensitivity, specificity, and positive predictive value. Secondary outcome was reduction in workload measured as arbitration rate and number of cases requiring second human reading.Results The novel workflow was found to be superior or non-inferior on all screening metrics, almost halving arbitration and reducing the number of cases requiring second human reading by up to 87.50% compared to human double reading.Conclusions AI as a supporting reader adds a safety net in case of AI discordance compared to alternative workflows where AI replaces the second human reader. In the simulation using large-scale historical data, the proposed workflow retains screening performance of the standard of care of human double reading while drastically reducing the workload. Further research should study the impact of the change in case mix for the second human reader as they would only assess cases where the AI and first human reader disagree.Competing Interest StatementA.Y.N., B.G., C.O., G.F., J.N., E.K., S.K., P.D.K. are employees of Kheiron Medical Technologies and hold stock options as part of the standard compensation package.Funding StatementThis work was funded by Kheiron Medical Technologies.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:The study had UK National Health Service Health Research Authority (REC reference 19/HRA/0376) and ETT-TUKEB Medical Research Council, Scientific and Research Ethics Committee, Hungary approval (reg no OGYEI/46651-4/2020).I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe data for the current study are not publicly available. Due to reasonable privacy and ethical concerns, the imaging data cannot be distributed to researchers without ethical approval and research agreements with the original data providers.