Abstract
Isolation of patients with communicable infectious diseases limits spread of pathogens but can be difficult to manage outside hospitals. Rwanda deployed a digital health service nationally to assist public health clinicians to remotely monitor and support SARS-CoV-2 cases via their mobile phones using daily interactive short message service (SMS) check-ins. We aimed to assess the texting patterns and communicated topics to understand patient experiences. We extracted data on all COVID-19 cases and exposed contacts who were enrolled in the WelTel text messaging program between March 18, 2020, and March 31, 2022, and linked demographic and clinical data from the national COVID-19 registry. A sample of the text conversation corpus was English-translated and labeled with topics of interest defined by medical experts. Multiple natural language processing (NLP) topic classification models were trained and compared using F1 scores. Best performing models were applied to classify unlabeled conversations. Total 33,081 isolated patients (mean age 33·9, range 0-100), 44% female, including 30,398 cases and 2,683 contacts) were registered in WelTel. Registered patients generated 12,119 interactive text conversations in Kinyarwanda (n=8,183, 67%), English (n=3,069, 25%) and other languages. Sufficiently trained large language models (LLMs) were unavailable for Kinyarwanda. Traditional machine learning (ML) models outperformed fine-tuned transformer architecture language models on the native untranslated language corpus, however, the reverse was observed of models trained on English-only data. The most frequently identified topics discussed included symptoms (69%), diagnostics (38%), social issues (19%), prevention (18%), healthcare logistics (16%), and treatment (8·5%). Education, advice, and triage on these topics were provided to patients. Interactive text messaging can be used to remotely support isolated patients in pandemics at scale. NLP can help evaluate the medical and social factors that affect isolated patients which could ultimately inform precision public health responses to future pandemics.
Author Summary We present the first application of NLP for categorizing text messages between patients and healthcare providers within a nationally scaled digital healthcare program. This study provides unique insights into the circumstances of home-based COVID-19 patients during the pandemic. Our trained topic classification models accurately categorized topics in both English and African language texts. Patients reported and discussed both medical and social issues with public healthcare providers. This approach has the potential to guide precision public health decisions and responses in future outbreaks, pandemics, and remote healthcare scenarios.
Competing Interest Statement
RTL is co-founder and chief scientific officer of WelTel Incorporated. WelTel Incorporated received a grant from Grand Challenges Canada to provide the software and technical support in Rwanda during the pandemic. No other authors have conflict of interests to declare. Data stewardship and reporting was overseen by the Rwanda Biomedical Center.
Funding Statement
This study was funded by the Canadian Institutes of Health Research (CIHR) and Grand Challenges Canada.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The Clinical Research Ethics Board of the University of British Columbia gave ethical approval for this work. The Rwanda National Ethics Committee affiliated with the Rwanda Biomedical Center and Rwanda Ministry of Health also gave ethical approval for this work.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
The WelTel software was installed and run on the Rwanda government National Data Center and using the government contracted API (FDI) to connect to cellular networks. The Rwanda government remains the steward of all patient data produced or collected by the WelTel platform and the COVID-19 national data registry. Deidentified text conversation training data sets can be made available on request. All NLP models used were open source and references for NLP tools are provided in the Methods and Supplementary Material.
