Abstract
Purpose Mammographic density is associated with the risk of developing breast cancer and can be predicted using deep learning methods. Model uncertainty estimates are not produced by standard regression approaches but would be valuable for clinical and research purposes. Our objective is to produce deep learning models with in-built uncertainty estimates without degrading predictive performance.
Approach We analyse data from over 150,000 mammogram images with associated continuous density scores from expert readers in the Predicting Risk Of Cancer At Screening (PROCAS) study. We re-designate the continuous density scores to 100 density classes then train classification and ordinal deep learning models. Distributions and distribution-free methods are applied to extract predictions and uncertainties. A deep learning regression model is trained on the continuous density scores to act as a direct comparison.
Results The root mean squared error (RMSE) between expert assigned density labels and predictions of the standard regression model are 8.42 (8.34-8.51) while the RMSE for the classification and ordinal classification are 8.37 (8.28-8.46) and 8.44 (8.35-8.53) respectively. The average uncertainties produced by the models are higher when the density scores from pairs of expert readers density scores differ more, are higher when different mammogram views of the same views are more variable and when two separately trained models show higher variation.
Conclusions Using either a classification or ordinal approach we can produce model uncertainty estimates without loss of predictive performance.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was funded by Canadian Institutes of Health Research (CIHR grant #169005). D. Gareth Evans and Susan M. Astley are supported by the National Institute for Health Research (NIHR) Manchester Biomedical Research Centre (Grant No. IS-BRC-1215-20007). Anne Martel is partially supported by the Tory Family Chair in Oncology.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics approval for the study was through the North Manchester Research Ethics Committee (09/H1008/81). Informed consent was obtained from all participants on entry to the PROCAS study.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
↵* E-mail: s.squires{at}exeter.ac.uk
Data Availability
Data is not currently available to researchers outside the University of Manchester.
