PT  - JOURNAL ARTICLE
AU  - Camacho-García-Formentí, Dalia
AU  - Baylón-Vázquez, Gabriela
AU  - Arriozola-Rodríguez, Karen
AU  - Avalos-Ramirez, Enrique
AU  - Hartleben-Matkin, Curt
AU  - Valdez-Flores, Hugo
AU  - Hodelin-Fuentes, Damaris
AU  - Noriega, Alejandro
TI  - Comparative Performance of retinIA, an AI-powered Ophthalmic Screening Tool, and First-Year Residents in Retinal Disease Detection and Glaucoma Assessment: A Study in a Mexican Tertiary Care Setting
AID  - 10.1101/2024.08.26.24311677
DP  - 2024 Jan 01
TA  - medRxiv
PG  - 2024.08.26.24311677
4099  - http://medrxiv.org/content/early/2024/08/28/2024.08.26.24311677.short
4100  - http://medrxiv.org/content/early/2024/08/28/2024.08.26.24311677.full
AB  - Background Artificial intelligence (AI) shows promise in ophthalmology, but its performance in diverse healthcare settings remains understudied. We evaluated retinIA, an AI-powered screening tool developed with Mexican data, against first-year ophthalmology residents in a tertiary care setting in Mexico City.Methods We analyzed 435 adult patients undergoing their first ophthalmic evaluation. RetinIA and residents’ assessments were compared against expert annotations for retinal lesions, cup-to-disk ratio (CDR) measurements, and glaucoma suspect detection. We also evaluated a synergistic approach combining AI and resident assessments.Results For glaucoma suspect detection, retinIA outperformed residents in accuracy (88.6% vs 82.9%, p = 0.016), sensitivity (63.0% vs 50.0%, p = 0.116), and specificity (94.5% vs 90.5%, p = 0.062). While, the synergistic approach deemed a higher sensitivity (80.4%) than ophthalmic residents alone or retinIA alone (p < 0.001). RetinIA’s CDR estimates showed lower mean absolute error (0.056 vs 0.105, p < 0.001) and higher correlation with expert measurements (r = 0.728 vs r = 0.538). In retinal lesion detection, retinIA demonstrated superior sensitivity (90.1% vs 63.0% for medium/high-risk lesions, p < 0.001) and specificity (95.8% vs 90.4%, p < 0.001). Furthermore, differences between retinIA and residents were statistically significant across all metrics. The synergistic approach achieved the highest sensitivity for retinal lesions (92.6% for medium/high-risk, 100% for high-risk) while maintaining good specificity (87.4%).Conclusion RetinIA outperforms first-year residents in key ophthalmic assessments. The synergistic use of AI and resident assessments shows potential for optimizing diagnostic accuracy, highlighting the value of AI as a supportive tool in ophthalmic practice, especially for earlycareer clinicians.Competing Interest StatementTwo of the authors, Dalia Camacho-García-Formentí and Alejandro Noriega, are employees of Prosperia Salud, the enterprise that developed retinIA. This relationship could be perceived as a potential conflict of interest. However, the study was conducted with the objective of maintaining scientific integrity and neutrality. All other authors declare no conflicts of interest.Funding StatementThis study did not receive any fundingAuthor DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:This project was approved by: Fundación de Asistencia Privada Conde de Valenciana IAP Ethics in Research Committee with registry number CONBIOÉTICA-09-CEI-023-20160830 and project approval number CEI-2023/12/01. Fundación de Asistencia Privada Conde de Valenciana IAP Biosecurity Committee with registry number 17 CB 09 015 007 and project approval number CB-0053-2023. The Fundación de Asistencia Privada Conde de Valenciana IAP Research Committee with registry number 17 CI 09 015 008 and project approval number CI-053-2023.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesAll data produced in the present study are available upon reasonable request to the authors.