Abstract
Importance Diagnosis of head and neck squamous dysplasias and carcinomas is challenging, with a moderate inter-rater agreement. Nowadays, new artificial intelligence (AI) models are developed to automatically detect and grade lesions, but their contribution to the performance of pathologists hasn’t been assessed.
Objective To evaluate the contribution of our AI tool in assisting pathologists in diagnosing squamous dysplasia and carcinoma in the head and neck region.
Design, Setting, and Participants We evaluated the effectiveness of our previously described AI model, which combines an automatic classification of laryngeal and pharyngeal squamous lesions with a confidence score, on a panel of eight pathologists coming from different backgrounds and with different levels of experience on a subset of 115 slides.
Main Outcomes and Measures The main outcome was the inter-rater agreement, measured by the weighted linear kappa. Other outcomes on diagnostic efficiency were assessed using paired t tests.
Results AI-Assistance significantly improved the inter-rater agreement (linear kappa 0.73, 95%CI [0.711-0.748] with assistance versus 0.675, 95%CI [0.579-0.765] without assistance, p < 0.001). The agreement was even better on high confidence predictions (mean linear kappa 0.809, 95%CI [0.784-0.834] for assisted review, versus 0.731, 95%CI [0.681-0.781] non-assisted, p = 0.018). These improvements were particularly strong for non-specialized and younger pathologists. Hence, the AI-Assistance enabled the panel to perform on par with the expert panel described in the literature.
Conclusions and Relevance Our AI-Assistance is of great value for helping pathologists in the difficult task of diagnosing squamous dysplasias and carcinomas, improving for the first time the inter-rater agreement. It demonstrates the possibility of a truly Augmented Pathology in complex tasks such as the classification of head and neck squamous lesions.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
ML was supported by a CIFRE PhD fellowship founded by Keen Eye, Paris, France and ANRT (CIFRE 2019/1905). Furthermore, this work was supported by the French government under management of Agence Nationale de la Recherche as part of the Investissements d avenir program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics Committee of Assistance Publique - Hopitaux de Paris Centre (CERAPHP. Centre - Institutional Review Board registration #00011928).
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
- Title - 95%CI
