Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

An independent, multi-country head-to-head accuracy comparison of automated chest x-ray algorithms for the triage of pulmonary tuberculosis

William Worodria, Robert Castro, Sandra V. Kik, Victoria Dalay, Brigitta Derendinger, Charles Festo, Thanh Quoc Nguyen, Mihaja Raberahona, Swati Sudarsan, Alfred Andama, Balamugesh Thangakunam, Issa Lyimo, Viet Nhung Nguyen, Rivo Rakotoarivelo, Grant Theron, Charles Yu, Claudia M. Denkinger, Simon Grandjean Lapierre, Adithya Cattamanchi, Devasahayam J. Christopher, Devan Jaganath, R2D2 TB Network
doi: https://doi.org/10.1101/2024.06.19.24309061
William Worodria
1World Alliance for Lung and Intensive Care in Uganda, Kampala, Uganda
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Robert Castro
2Division of Pulmonary and Critical Care Medicine, University of California, San Francisco, San Francisco, USA
3Center for Tuberculosis, University of California, San Francisco, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sandra V. Kik
4FIND, Geneva, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Victoria Dalay
5De La Salle Medical and Health Sciences Institute, Dasmarinas Cavite, Philippines
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Brigitta Derendinger
6DSI-NRF Centre of Excellence for Biomedical Tuberculosis Research, South African Medical Research Council Centre for Tuberculosis Research, Division of Molecular Biology and Human Genetics, Faculty of Medicine and Health Sciences, Stellenbosch University, South Africa
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Charles Festo
7Ifakara Health Institute, Dar es Salaam, Tanzania
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Thanh Quoc Nguyen
8Vietnam National Tuberculosis Programme, National Lung Hospital, Hanoi, Vietnam
9VNU University of Medicine and Pharmacy, Hanoi, Vietnam
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mihaja Raberahona
10Department of Infectious Diseases, CHU Joseph Raseta Befelatanana, Antananarivo, Madagascar
11Centre d’Infectiologie Charles Mérieux, Université d’Antananarivo, Antananarivo, Madagascar
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Swati Sudarsan
2Division of Pulmonary and Critical Care Medicine, University of California, San Francisco, San Francisco, USA
3Center for Tuberculosis, University of California, San Francisco, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alfred Andama
1World Alliance for Lung and Intensive Care in Uganda, Kampala, Uganda
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Balamugesh Thangakunam
12Department of Pulmonary Medicine, Christian Medical College, Vellore, Tamil Nadu, India
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Issa Lyimo
7Ifakara Health Institute, Dar es Salaam, Tanzania
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Viet Nhung Nguyen
8Vietnam National Tuberculosis Programme, National Lung Hospital, Hanoi, Vietnam
9VNU University of Medicine and Pharmacy, Hanoi, Vietnam
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rivo Rakotoarivelo
11Centre d’Infectiologie Charles Mérieux, Université d’Antananarivo, Antananarivo, Madagascar
13Faculté de Médecine, Université de Fianarantsoa, Fianarantsoa, Madagascar
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Grant Theron
6DSI-NRF Centre of Excellence for Biomedical Tuberculosis Research, South African Medical Research Council Centre for Tuberculosis Research, Division of Molecular Biology and Human Genetics, Faculty of Medicine and Health Sciences, Stellenbosch University, South Africa
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Charles Yu
5De La Salle Medical and Health Sciences Institute, Dasmarinas Cavite, Philippines
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Claudia M. Denkinger
14Department of Infectious Disease and Tropical Medicine, Center for Infectious Diseases, Heidelberg University Hospital, Heidelberg, Germany
15German Center for Infection Research, partner site, Heidelberg, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Simon Grandjean Lapierre
16Centre de Recherche du Centre Hospitalier de l’Université de Montréal, Immunopathology Axis, Montréal, Canada
17Université de Montréal, Department of Microbiology, Infectious Diseases and Immunology, Montréal, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Adithya Cattamanchi
3Center for Tuberculosis, University of California, San Francisco, USA
18Division of Pulmonary Diseases and Critical Care Medicine, School of Medicine, University of California Irvine, Orange, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: acattama{at}hs.uci.edu
Devasahayam J. Christopher
10Department of Infectious Diseases, CHU Joseph Raseta Befelatanana, Antananarivo, Madagascar
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Devan Jaganath
3Center for Tuberculosis, University of California, San Francisco, USA
19Division of Pediatric Infectious Diseases, University of California, San Francisco, San Francisco, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

ABSTRACT

Background Computer-aided detection (CAD) algorithms for automated chest X-ray (CXR) reading have been endorsed by the World Health Organization for tuberculosis (TB) triage, but independent, multi-country assessment and comparison of current products are needed to guide implementation.

Methods We conducted a head-to-head evaluation of five CAD algorithms for TB triage across seven countries. We included CXRs from adults who presented to outpatient facilities with at least two weeks of cough in India, Madagascar, the Philippines, South Africa, Tanzania, Uganda, and Vietnam. The participants completed a standard evaluation for pulmonary TB, including sputum collection for Xpert MTB/RIF Ultra and culture. Against a microbiological reference standard, we calculated and compared the accuracy overall, by country and key groups for five CAD algorithms: CAD4TB (Delft Imaging), INSIGHT CXR (Lunit), DrAid (Vinbrain), Genki (Deeptek), and qXR (qure.AI). We determined the area under the ROC curve (AUC) and if any CAD product could achieve the minimum target accuracy for a TB triage test (≥90% sensitivity and ≥70% specificity). We then applied country- and population-specific thresholds and recalculated accuracy to assess any improvement in performance.

Results Of 3,927 individuals included, the median age was 41 years (IQR 29-54), 12.9% were people living with HIV (PLWH), 8.2% living with diabetes, and 21.2% had a prior history of TB. The overall AUC ranged from 0.774-0.819, and specificity ranged from 64.8-73.8% at 90% sensitivity. CAD4TB had the highest overall accuracy (73.8% specific, 95% CI 72.2-75.4, at 90% sensitivity), although qXR and INSIGHT CXR also achieved the target 70% specificity. There was heterogeneity in accuracy by country, and females and PLWH had lower sensitivity while males and people with a history of TB had lower specificity. The performance remained stable regardless of diabetes status. When country- and population-specific thresholds were applied, at least one CAD product could achieve or approach the target accuracy for each country and sub-group, except for PLWH and those with a history of TB.

Conclusions Multiple CAD algorithms can achieve or exceed the minimum target accuracy for a TB triage test, with improvement when using setting- or population-specific thresholds. Further efforts are needed to integrate CAD into routine TB case detection programs in high-burden communities.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

The R2D2 TB Network was supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under award number U01AI152087, and the Digital Cough Monitoring study was funded by the Patrick J. McGovern Foundation. SGL is supported by a Junior 1 Salary Award from the Fonds de Recherche Sante Quebec. DJ is supported by funding by the National Institutes of Health (K23HL153581). GT acknowledges funding from the EDCTP2 programme supported by the European Union (RIA2018D-2509, PreFIT; RIA2018D-2493, SeroSelectTB; RIA2020I-3305, CAGE-TB) and the National Institutes of Health (D43TW010350; U01AI152087; U54EB027049; R01AI136894).

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

Ethical review boards from Christian Medical College (Vellore, India), De La Salle Medical and Health Sciences Institute (Dasmarinas City, Philippines), Stellenbosch University (Cape Town, South Africa), Makerere University College of Health Sciences (Kampala, Uganda), the National Lung Hospital (Hanoi, Vietnam), Comite d'Ethique a la Recheche Biomedicale (Antananarivo, Madagascar), Ifakara Health Institute (Ifakara, Tanzania), the Centre de Recherche du Centre Hospitalier de l'Universite de Montreal (Montreal, Canada) and the University of California, San Francisco (San Francisco, USA) gave ethical approval for this work.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Data Availability

All data produced in the present study are available upon reasonable request to the authors

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted June 19, 2024.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
An independent, multi-country head-to-head accuracy comparison of automated chest x-ray algorithms for the triage of pulmonary tuberculosis
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
An independent, multi-country head-to-head accuracy comparison of automated chest x-ray algorithms for the triage of pulmonary tuberculosis
William Worodria, Robert Castro, Sandra V. Kik, Victoria Dalay, Brigitta Derendinger, Charles Festo, Thanh Quoc Nguyen, Mihaja Raberahona, Swati Sudarsan, Alfred Andama, Balamugesh Thangakunam, Issa Lyimo, Viet Nhung Nguyen, Rivo Rakotoarivelo, Grant Theron, Charles Yu, Claudia M. Denkinger, Simon Grandjean Lapierre, Adithya Cattamanchi, Devasahayam J. Christopher, Devan Jaganath, R2D2 TB Network
medRxiv 2024.06.19.24309061; doi: https://doi.org/10.1101/2024.06.19.24309061
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
An independent, multi-country head-to-head accuracy comparison of automated chest x-ray algorithms for the triage of pulmonary tuberculosis
William Worodria, Robert Castro, Sandra V. Kik, Victoria Dalay, Brigitta Derendinger, Charles Festo, Thanh Quoc Nguyen, Mihaja Raberahona, Swati Sudarsan, Alfred Andama, Balamugesh Thangakunam, Issa Lyimo, Viet Nhung Nguyen, Rivo Rakotoarivelo, Grant Theron, Charles Yu, Claudia M. Denkinger, Simon Grandjean Lapierre, Adithya Cattamanchi, Devasahayam J. Christopher, Devan Jaganath, R2D2 TB Network
medRxiv 2024.06.19.24309061; doi: https://doi.org/10.1101/2024.06.19.24309061

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Public and Global Health
Subject Areas
All Articles
  • Addiction Medicine (349)
  • Allergy and Immunology (668)
  • Allergy and Immunology (668)
  • Anesthesia (181)
  • Cardiovascular Medicine (2648)
  • Dentistry and Oral Medicine (316)
  • Dermatology (223)
  • Emergency Medicine (399)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (942)
  • Epidemiology (12228)
  • Forensic Medicine (10)
  • Gastroenterology (759)
  • Genetic and Genomic Medicine (4103)
  • Geriatric Medicine (387)
  • Health Economics (680)
  • Health Informatics (2657)
  • Health Policy (1005)
  • Health Systems and Quality Improvement (985)
  • Hematology (363)
  • HIV/AIDS (851)
  • Infectious Diseases (except HIV/AIDS) (13695)
  • Intensive Care and Critical Care Medicine (797)
  • Medical Education (399)
  • Medical Ethics (109)
  • Nephrology (436)
  • Neurology (3882)
  • Nursing (209)
  • Nutrition (577)
  • Obstetrics and Gynecology (739)
  • Occupational and Environmental Health (695)
  • Oncology (2030)
  • Ophthalmology (585)
  • Orthopedics (240)
  • Otolaryngology (306)
  • Pain Medicine (250)
  • Palliative Medicine (75)
  • Pathology (473)
  • Pediatrics (1115)
  • Pharmacology and Therapeutics (466)
  • Primary Care Research (452)
  • Psychiatry and Clinical Psychology (3432)
  • Public and Global Health (6527)
  • Radiology and Imaging (1403)
  • Rehabilitation Medicine and Physical Therapy (814)
  • Respiratory Medicine (871)
  • Rheumatology (409)
  • Sexual and Reproductive Health (410)
  • Sports Medicine (342)
  • Surgery (448)
  • Toxicology (53)
  • Transplantation (185)
  • Urology (165)