Abstract
Background Convolutional neural networks (CNNs) have been used to classify medical images; few studies use smartphone photographs that are scalable at point of care. We previously showed proof of principle that CNNs could detect inflammatory arthritis in three hand joints. We now studied a screening CNN to differentiate from controls.
Methods We studied consecutive patients with early inflammatory arthritis and healthy controls, all examined by a rheumatologist (15% by two). Standardized photographs of the hands were taken using a studio box, anonymized, and cropped around joints. We fine-tuned pre-trained CNN models on our dataset (80% training; 20% test set). We used an Inception-ResNet-v2 backbone CNN modified for two class outputs (Patient vs Control) on uncropped photos. Inception-ResNet-v2 CNNs were trained on cropped photos of Middle finger Proximal Interphalangeal (MFPIP), Index finger PIP (IFPIP) and wrist. We report representative values of accuracy, sensitivity, specificity.
Results We studied 800 hands from 200 controls (mean age 37.8 years) and 200 patients (mean age 49.6 years; 134 with rheumatoid arthritis amongst other diagnoses). Two rheumatologists had a concordance of 0.89 in 404 joints. The wrist was commonly involved (173/400) followed by the MFPIP (134) and IFPIP (128). The screening CNN achieved excellent accuracy (98%), sensitivity (98%) and specificity (98%) in predicting a patient compared to controls. Joint-specific CNN accuracy, sensitivity and specificity were highest for the wrist (80%, 88%, 72%) followed by the IFPIP (79%, 89%,73%) and MFPIP (76%, 91%, 70%).
Conclusion Computer vision without feature engineering can distinguish between patients and controls based on smartphone photographs with good accuracy, showing promise as a screening tool prior to joint-specific CNNs. Future research includes validating findings in diverse populations, refining models to improve specificity in joints and integrating this technology into clinical workflows.
Competing Interest Statement
SP and PG are coinventors on a provisional patent application at the Indian Patent Office (awaiting examination for final) that includes some of the material used in this manuscript. SP is co-founder at Med2Measure Pvt. Ltd, that deals with remote detection of arthritis. The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Funding Statement
This study did not receive any funding
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics committee of KEM Hospital Research Centre gave ethical approval for this work.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data produced in the present study are available upon reasonable request to the authors
