Abstract
Objectives To evaluate the effectiveness of a novel strategy for using AI as a supporting reader for the detection of breast cancer in mammography-based double reading screening practice. Instead of replacing a human reader, here AI serves as the second reader only if it agrees with the recall/no-recall decision of the first human reader. Otherwise, a second human reader makes an assessment, enacting standard human double reading.
Design Retrospective large-scale, multi-site, multi-device, evaluation study.
Participants 280,594 cases from 180,542 female participants who were screened for breast cancer with digital mammography between 2009 and 2019 at seven screening sites in two countries (UK and Hungary).
Main outcome measures Primary outcome measures were cancer detection rate, recall rate, sensitivity, specificity, and positive predictive value. Secondary outcome was reduction in workload measured as arbitration rate and number of cases requiring second human reading.
Results The novel workflow was found to be superior or non-inferior on all screening metrics, almost halving arbitration and reducing the number of cases requiring second human reading by up to 87.50% compared to human double reading.
Conclusions AI as a supporting reader adds a safety net in case of AI discordance compared to alternative workflows where AI replaces the second human reader. In the simulation using large-scale historical data, the proposed workflow retains screening performance of the standard of care of human double reading while drastically reducing the workload. Further research should study the impact of the change in case mix for the second human reader as they would only assess cases where the AI and first human reader disagree.
Competing Interest Statement
A.Y.N., B.G., C.O., G.F., J.N., E.K., S.K., P.D.K. are employees of Kheiron Medical Technologies and hold stock options as part of the standard compensation package.
Funding Statement
This work was funded by Kheiron Medical Technologies.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The study had UK National Health Service Health Research Authority (REC reference 19/HRA/0376) and ETT-TUKEB Medical Research Council, Scientific and Research Ethics Committee, Hungary approval (reg no OGYEI/46651-4/2020).
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
The data for the current study are not publicly available. Due to reasonable privacy and ethical concerns, the imaging data cannot be distributed to researchers without ethical approval and research agreements with the original data providers.
