Abstract
Purpose To fine tune and evaluate the performance of the retinal foundation model (RETFound) on a diverse longitudinal clinical research dataset in glaucoma detection from optical coherence tomography (OCT) RNFL scans. Subanalyses of the model performance were evaluated across different subgroups, various dataset sample sizes and training cycles (epochs).
Design Evaluation of a diagnostic technology
Subjects, Participants, and Controls 15,216 Spectralis OCT RNFL circle scans of 747 individuals of diverse race (56.9% White, 37.8% Black/African American, and 5.3% Other/Not reported, glaucoma severity (30.8% mild, 18.4% moderate-to-severe, and 50.9% no glaucoma), and age (44.8% <60 years, 55.2% >60 years) from the Diagnostic Innovations in Glaucoma Study (DIGS) and the African Descent and Glaucoma Evaluation Study (ADAGES). All OCT scans were labeled as “Non-glaucomatous” or “Glaucomatous.“
Methods RETFound was employed to perform binary glaucoma classification. The diagnostic accuracy of RETFound was iteratively tested across different combinations of dataset sample sizes (50 to 2000 OCT RNFL circle scans), epochs (5 to 50), and study subpopulations stratified by severity of glaucoma, age, and race).
Main Outcome Measures Area under receiver operating characteristic curve (AUC) for classifying RNFL scans as “Non-glaucomatous” or “Glaucomatous.“
Results Performance metrics improved with larger training datasets and more training cycles, rising from an AUC of 0.61 (50 training images and 5 epochs) to AUC 0.91 (2,000 training images and 50 epochs). Gains in performance were marginal as training size increased beyond 500 scans. Performance was similar across race for all training size and cycle number combinations: African American (AUC=0.90) vs other (AUC=0.93). RNFL scans from older patients (>60 years) led to worse performance (AUC=0.85) compared to younger patients (<60 years, AUC=0.95). Performance was significantly higher for RNFL scans from patients with moderate-to-severe glaucoma vs mild glaucoma (AUC=0.99 vs 0.88, respectively).
Conclusions Good RETFound performance was observed with a relatively small sample size of images used for fine tuning and across differences in race and age. RETFound’s ability to adapt across a range of OCT training conditions and populations suggests it is a promising tool to automate glaucoma detection in a variety of use cases.
Precis The study found high accuracy for glaucoma detection from OCT optic nerve head RNFL scans in a diverse study population by adapting an existing foundation model (RETFound). Performance improved with larger datasets and more training cycles, achieving an AUC of 0.91 with RNFL scans alone. Results suggest RETFound is promising for automated OCT RNFL-based glaucoma detection across demographics and training conditions.
Competing Interest Statement
Benton Chuter and Justin Huynh report no conflicts, with Huynh only listing the T35 grant from the National Research Service Award (NRSA). Linda M. Zangwill discloses support from the National Eye Institute, The Glaucoma Foundation, National Institutes of Health, and Heidelberg Engineering, along with consulting fees from AbbVie and Topcon Medical Systems, and involvement with AISight Health Inc. Evan Walker lists grants from the National Eye Institute and P30EY022589. Christopher A. Girkin discloses funding from the National Eye Institute, Topcon, EyeSight Foundation of Alabama, Research to Prevent Blindness, and Heidelberg Engineering. Mark Christopher and Robert N. Weinreb also acknowledge support from the National Eye Institute and other institutions, with Weinreb additionally reporting equipment support from Centervue, Optovue, and Topcon Medical. Massimo Fazio details funding from the National Eye Institute, EyeSight Foundation of Alabama, Research to Prevent Blindness, Heidelberg Engineering GmbH, and Topcon Healthcare Inc., with research support from Wolfram Research Inc. Shahin Hallaj reports a stipend from the NIH Bridge2AI common fund. Jalil Jalili discloses no conflicts.
Funding Statement
Benton Chuter lists support from UCSD MEDGAP 2023. Linda M. Zangwill discloses funding from the National Eye Institute, The Glaucoma Foundation, National Institutes of Health, and Heidelberg Engineering. Evan Walker lists grants from the National Eye Institute and P30EY022589. Christopher A. Girkin discloses support from the National Eye Institute, Topcon, EyeSight Foundation of Alabama, Research to Prevent Blindness, and Heidelberg Engineering. Mark Christopher acknowledges funding from the National Eye Institute, The Glaucoma Foundation, K99EY030942, and R00EY030942. Massimo Fazio details support from the National Eye Institute, EyeSight Foundation of Alabama, Research to Prevent Blindness, Heidelberg Engineering GmbH, Topcon Healthcare Inc., and research support from Wolfram Research Inc. Shahin Hallaj reports a stipend from the NIH Bridge2AI common fund. Robert N. Weinreb discloses funding from Topcon Medical, the National Institutes of Health, Research to Prevent Blindness, and equipment support from Centervue, Optovue, and Topcon Medical.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The recruitment process and methodology were approved by the institutional review boards at each participating site, adhering to the ethical standards outlined in the Declaration of Helsinki and the Health Insurance Portability and Accountability Act. All subjects provided informed consent during recruitment. IRB of The University of California San Diego Hamilton Glaucoma Center and Viterbi Family Department of Ophthalmology gave ethical approval for this work. IRB of The University of Alabama at Birmingham Department of Ophthalmology gave ethical approval for this work. IRB of The Columbia University Medical Center Edward S. Harkness Eye Institute gave ethical approval for this work.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
Full results described in the present study are available upon reasonable request to the authors. Public access to images and associated clinical data sets is limited.
