Abstract
Background Despite the recent proliferation of digital health technologies (DHTs), there is a lack of formal, industry-wide standards to evaluate the performance of the product’s algorithm in terms of its ability to measure, detect, or predict a clinical state. The advancement and successful use of DHTs in medicine requires that all stakeholders – clinicians, patients, payers, regulators, pharmaceutical companies, and the medical products industry – have a common understanding of what it means when a DHT has been analytically validated.
Objective We conducted a systematic review to assess the state of the science on analytical validation for DHTs, using the criteria established by the V3 Framework and EVIDENCE checklist.
Methods The systematic review was conducted according to the PRISMA guidelines. A multi-tier PubMed search identified studies published between April 15, 2020, and April 15, 2023, on analytical validation of DHTs; thereafter, each paper was assessed against the EVIDENCE checklist items specific to analytical validation. All studies were required to report quantitative data collected from a connected, mobile sensor product for the measurement, diagnosis, and/or treatment of a behavioral or physiological function, and compare the outcome measures to an established reference standard.
Results Of the 1201 papers identified in the literature search, we identified 303 reporting the results of a DHT analytical validation study. The most prevalent therapeutic areas of focus were neurological (26%), cardiovascular (18%), and sleep conditions (17%). Health outcome categories most frequently captured by DHTs were gait (15%), heart rate/rhythm (15%), blood pressure and/or arterial stiffness (11%), sleep staging (10%), and mobility (9%). Only 208 papers (69%) reported all components of the EVIDENCE checklist focused on analytical validation, with the exception of software version and race/ethnicity, thereby meeting our definition of high-quality evidence reporting.
Conclusion We are encouraged by the emerging literature evaluating whether outcome measures assessed by DHTs adequately reflect the physiological or behavioral parameter of interest; however, the quality of reporting is not yet sufficient to ensure the advancement of digital clinical measures that are fit-for-purpose for all members of a defined population and eliminate the need for redundant studies. We recommend that journals publishing analytical validation studies require the use of the EVIDENCE checklist as a reporting standard for these manuscripts.
Competing Interest Statement
JPB reports income and/or other financial interests in Apnimed, Koneksa Health, Philips, and Signifier Medical Technologies.
Funding Statement
This study did not receive any funding.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
Data are available upon request.
Abbreviations
- AV
- Analytical Validation
- BioMeTs
- Biometric Monitoring Technologies
- CONSORT
- Consolidated Standards of Reporting Trials
- DATAcc
- Digital Health Measurement Collaborative Community
- EVIDENCE
- EValuatIng connecteD sENsor teChnologiEs
- IEEE
- Institute of Electrical and Electronics Engineers
- PRISMA
- Preferred Reporting Items for Systematic Review and Meta-Analysis
- V3
- Verification, analytical validation and clinical validation
