Implementation of risk triaging in primary healthcare facilities in Sub-Saharan Africa: A systematic review

INTRODUCTION

One of the challenges facing treatment programs for HIV and other chronic conditions in sub-Saharan Africa is how to target interventions to maximize long-term retention in care [1–4]. Ideally, interventions aimed at improving adherence and retention should be offered to patients at higher risk of disengagement from care or poor adherence, while not adding to the burden of care or utilizing resources unnecessarily for lower risk patients who do not require additional attention. Identifying high risk patients before they experience negative outcomes, however, remains a puzzle.

Traditionally, most efforts to target interventions aimed at improving retention in care or reducing interruptions in treatment have identified high risk patients based on observed behaviour after negative events have occurred. Patients who are observed to miss clinical visits or medication refills, for example, are identified after the event and targeted for tracing, counselling, home-based care, and other services that may help them resume scheduled care [5–10]. An alternative to this post-hoc strategy for identifying high risk patients is “risk triaging,” or the process of identifying vulnerable or higher risk patients before they experience an interruption in care. Predictive models developed into risk scoring or triaging tools have a long history of use in hospitals, in high income countries, and for non-communicable conditions [11–13].

While risk triaging or risk scoring can be a straightforward approach for evaluating clinical symptoms, gauging a patient’s risk of a behaviour such as disengaging from chronic disease care is much more challenging. Some broad (and largely unmodifiable) patient characteristics, such as sex and age, have consistently been associated with higher loss to follow up rates in the literature[14,15] but these do not identify subgroups that allow for practical or efficient targeting of interventions. Recognizing this, multiple attempts have been made to identify patients who would benefit from early intervention. These include tools developed to identify patients at risk of ART default and poor viral load outcomes [16–19]; patients qualifying for same day ART initiation [20,21]; adults and children in need of HIV testing [22,23]; and patients likely to return to care after disengagement [24]Results of these tools have been mixed, in terms of both accuracy and uptake by healthcare providers. While much risk triaging is conducted in hospitals and other high-care settings, it is at primary healthcare clinics that most patients in SSA initiate and sustain ART. To our knowledge, the only example of HIV-related risk triaging commonly used at the primary healthcare clinic level in SSA is the World Health Organization’s symptom screen for tuberculosis, which triages patients with cough, fever, night sweats, or weight loss for TB diagnostic tests [25–27].

The absence of risk triaging for patients on ART precludes providers from being able to target specific types of support to specific patients who need that support. Patients who anticipate that transport fares will pose an obstacle to visit attendance, for example, could be offered different interventions from those who fear accidental disclosure of their HIV status, if these obstacles were identified in advance. As part of an effort to design differentiated models of HIV treatment delivery for patients in their first six months after antiretroviral therapy initiation, we are developing a risk triaging tool that could be used by clinical and lay providers to match retention interventions to levels and types of risks[28,29]. To lay the groundwork for this effort, we conducted a systematic review of the use of risk triaging tools at the primary healthcare level in SSA. We asked two main questions: 1) Can risk triaging tools accurately stratify patients into higher and lower risk groups in primary healthcare settings? and 2) How has risk triaging been implemented in primary healthcare facilities in SSA?

METHODS

RESULTS

Search results

Our search strategy yielded a total of 1,876 articles from the six databases searched (Figure 1). We removed 107 articles duplicated across databases and screened abstracts and titles from 1,769 articles. Of these, 42 articles met the inclusion criteria. Of 1,708 articles that did not meet the inclusion criteria, some were excluded for multiple reasons, but having the wrong study design (n=1,201), no evidence of a risk triaging tool being implemented in the study (n=989), or involving the wrong study population (n= 745) were the most frequent reasons for article exclusion. From abstracts, we also identified 19 published systematic reviews to include in the snowballing process. We reviewed the reference lists of these articles and identified a further 40 articles meeting our inclusion criteria, for a total of 82 articles extracted for full text review. We excluded 54 of these; half of these exclusions were due to either study locations outside sub-Saharan Africa (n=14) or not in a primary health setting (n=12).

• Download figure
• Open in new tab

Figure 1: PRISMA flow chart of article screening process

The final data set included 28 articles (Table 1) that reported on risk triaging tools from 14 observational cohorts, 6 clinical trials, and 8 other study designs, with data collected and analysed between 2004 and 2020. The most frequent implementation countries were South Africa (n=12), Kenya (n=7), and Uganda (n=6); 8 were multi-country studies. Risk triaging tools were implemented in a diverse range of facility settings within the context of primary health care, including health facility-based outpatient clinics, community-based sites, and dedicated research facilities

View this table:

• View inline
• View popup

Table 1: Description of risk scoring and triaging tools

Characterization of risk triaging tools evaluated

Table 1 and Figure 2 describe the risk triaging tools identified in our search. Of the 28 studies, 13 addressed HIV, 10 TB, 1 TB/HIV, and 4 other conditions. In total, 24 (86%) of the triaging tools generated risk scores, while the remaining 4 utilized clinical algorithms, symptom screens, or clinical checks and diagnostics. The risk score approach was most frequently applied to HIV (n=13) and TB (n=8), while the other triaging approaches were applied to HIV and TB combined, sexually transmitted diseases, cancer, and mental health conditions.

• Download figure
• Open in new tab

Figure 2. Characterization of triaging tools stratified by condition for which the tool was developed (n=28)

The studies included in the review fell into two methodological categories. First, many studies used existing, retrospective data from cohort studies (n=9) or clinical trials (n=8) to identify important risk factors for an outcome and then used these to construct a scoring system. The published reports for these tools show no evidence of actual administration of the tool to a patient cohort and appear not to have been implemented at all. A second approach was to design a tool in advance—presumably based on existing data and/or clinical experience—and then report a study in which the tool was implemented in a trial or observational cohort setting (n=11). Most of the studies in both categories developed new tools, but others compared an existing or adapted tool to standard of care or to one or more other tools, such as comparing WHO’s latest TB symptom screening algorithm to its previous version. The tools that were administered in implementation studies used digital and/or paper formats for data collection; most did not report the format utilized.

Performance metrics

Performance metrics for the risk triaging tools evaluated are detailed in Table 2. Sensitivity of the applied tools varied between studies but also within studies that considered multiple thresholds were considered. More than half (n=16; 57%) achieved sensitivities >80%. As would be expected for risk triaging tools (in contrast to diagnostic tools), results for specificity were generally low. Of the 20 articles that reported specificity, 8 (40%) had a specificity <50%. Not all studies reported measures for negative and positive predictive value; those that did indicated negative predictive values ranging from 4%-99% and positive predictive values ranging from 1%-99%. Area under the curve (AUC) was also not universally reported, but for studies where this metric was available (n=16), most AUC values were >0.7, indicating that the tools correctly identified the outcome of interest in 7 or more of 10 patients.

View this table:

• View inline
• View popup

Table 2: Performance metrics of risk scoring tools

Figure 3 illustrates the summary performance of tools evaluated in the included studies. For each tool, we plotted a solid line connecting the point estimate for sensitivity with the estimate for false positive rate (1-specificity). Next NPV, PPV, and AUC (where available) were plotted as individual points around the plotted line. In this way, a tool with high performance metrics would be visualised as a wide solid line connecting point estimates for sensitivity and false positive rate with AUC and other metrics clustered near the right side of the axis. Several tools demonstrated these patterns[32,36]. There was a frequently observed trade-off, however, between sensitivity and specificity. Three TB screening algorithms assessed in Kenya, for example, demonstrated sensitivity between 73-91%, while specificity was <50% for all three [50]. The effect of outcome prevalence on PPV and NPV was also observed, such as for the VOICE risk scoring tool predicting risk of HIV acquisition[39].

• Download figure
• Open in new tab

Figure 3. Summary performance metrics of risk triaging tools*

*For each study, the solid line represents a plotted line connecting the point estimate for sensitivity with the point estimate for false positive rate (1-specificity) for that specific tool. Where available, negative predictive value (NPV), positive predictive value (PPV), and area under the curve (AUC) were plotted as individual points around the plotted line as indicated in the legend.

DISCUSSION

In settings that face a scarcity of resources—a situation that prevails in many public sector healthcare settings globally—identifying individuals at higher risk of negative outcomes now or in the future is a critical step in ensuring that available resources are allocated where they can do most good. While risk triaging is undoubtedly conducted informally by most if not all healthcare providers, based on their own experience and judgment, widespread use of clinical triaging tools appears to be rare in primary healthcare clinic settings in sub-Saharan Africa. There is even less use of structured instruments to identify future behavioural risks, such as attrition from care. In resource-scarce settings, many facilities are high-volume, staff-constrained sites, where effective triaging tools could improve the quality of clinical care while also increasing the efficiency of health care resource utilization.

The systematic review reported here is, to our knowledge, the first attempt to synthesize experience in developing, testing, and, in a few cases, implementing risk triaging tools in these settings. Our findings offer a mixed message, in terms of the future role of risk triaging in routine care. On one hand, most of the tools we identified were at least somewhat successful in identifying potential risk, with the majority of tools correctly identifying the outcome of interest in upwards of 70% of screened patients. The relatively low specificity of most of the tools is not a serious concern, since most were developed as screening tools rather than diagnostics, making high sensitivity the priority. For purposes of identifying groups of patients who are likely to benefit from additional testing or monitoring, versus those who clearly have no need for additional services, even a tool with modest sensitivity and low specificity has the potential to improve resource allocation, particularly in settings where prevalence of the outcome or condition screened for is high. In addition, an encouraging proportion of the tools we reviewed were explicitly designed to rely on indicators readily available at the primary healthcare level, rather than laboratory tests or imaging assays, though many also emphasized risk factors that are not easily addressed through practical interventions, such as age and sex.

On the other hand, based on the published reports, very few of the risk triaging tools identified were ever implemented beyond their initial studies or scaled up outside the research setting. Some developed and internally validated their risk scores using retrospective data and some tested risk triaging tools in small study populations, but none reported routine adoption of the prediction tools. There are likely multiple reasons for this, ranging from lack of logistical feasibility (e.g. an instrument required too much data to be feasible to administer during a routine clinical encounter) to lack of dissemination on the part of the developers. Without a clearly demonstrated need and demand for a triaging approach, as well as motivated and empowered champions willing and able to advocate for scale-up, many potentially well-performing and useful tools will not be utilised beyond their development setting. The lack of published information about how improvements to routine procedures, such as a new risk triaging tool, can be adopted at large scale is a gap in our understanding of how to improve primary healthcare performance.

While few of the tools we reviewed appear to have been adopted for widespread use, there is one (and perhaps only one) that has: the WHO’s TB symptom screening tool. The four- or five-item screening questionnaire is in widespread, if not near-universal, use throughout sub-Saharan Africa for people living with HIV. Several of the studies included here evaluated the TB symptom screen, alone or in combination with other indicators, and generated mixed results in terms of sensitivity and specificity for different populations. Exploration of how the TB symptom screen came to be so widely accepted and utilized suggests that a combination of meta-analysis of data from many sources, early and substantial engagement of policy makers, and international agency leadership allowed the TB symptom screen first to be incorporated into WHO guidelines[61] and then integrated into national guidelines and practice throughout the region. Better understanding of this process may be useful in introducing other risk triaging tools into widespread use.

Half of the studies included in our review addressed risks associated with HIV, whether transmission, treatment initiation, viral suppression, or mortality, and another third focused on TB case-finding. This may be explained by the very large numbers of individuals requiring these services in many sub-Saharan African countries. It may also reflect the large amounts of funding that have been available for HIV- and TB-related research and interventions in general, compared to other conditions that could be amenable to risk triaging, such as NCDs. Interestingly, however, none of the HIV-related studies included here offered a risk triaging tool aimed directly at adherence to HIV treatment, with retention in care as the outcome of interest. Researchers may have had more success in assessing risks of HIV acquisition, readiness for treatment, viral suppression, and mortality than in finding consistent risk predictors of the broader outcome of “retention,” but it is retention that remains the single most difficult challenge for national HIV programs. Continued research in this area may thus be warranted.

The studies reviewed here do provide a number of general and specific lessons about the characteristics of a promising risk triaging tool and how such a tool may be adopted into widespread use. First, tools must be brief and easy to integrate into routine care without disrupting established practices and rely on risk factors that are either already routinely recorded or are easy and quick to collect. Expanding existing data collection requirements is simply not feasible in busy, understaffed clinics where computer access, and even electricity, cannot be taken for granted. In the authors’ experience, moreover, primary healthcare clinic staff may resist innovations that they believe will increase their workload or complicate their tasks, even if the ultimate effect may be the opposite, such as an intervention that increases the number of candidates for early screening but reduces long-term treatment demand. Convincing healthcare staff that a new tool is worth the effort is likely to be much more successful if the tool itself is simple to understand and implement. Tools that can be implemented, wholly or in part, by lay cadres such as counsellors, peer supporters or community healthcare workers, rather than solely by scarcer cadres such as nurses, may also be more promising for widespread adoption, particularly as provision of primary care move towards community-based and other out-of-facility service delivery models.

Second, where risk scores are generated and utilised, interpretation of the output scores should be simple and straightforward. Decision cut-off points (e.g. having a risk score above or below a threshold) must be clear and well justified. We also note that while none of the studies we reviewed discussed offering healthcare providers discretion in interpreting risk scores, it seems reasonable that these scores should be seen as guidelines rather than absolute rules. If a clinician interacting with a patient believes that the risk score generated does not lead to the correct management pathway, clinician judgment may be preferred, as it is in many clinical situations. Similarly, individual patients may have strong reasons for preferring one pathway to another. Since patient cooperation is essential to achieving good outcomes, such preferences should be taken into account, no matter what a risk triaging tool suggests.

Third, in most cases, assessing risk is not an end in itself. It does little good, for example, to designate an HIV treatment patient as having a low or high risk of treatment interruption if there is not a clear pathway for responding to the conditions that create the assessed level of risk. A client who is determined to be at high risk of disengagement due to fears of disclosure and stigma will require an entirely different intervention strategy from a client at high risk of disengagement due to an employment situation that does not allow clinic visits during working hours. Risk triaging tools should thus come with intervention strategies or programs designed for the specific risk factors that contribute to the score or other outcome.

Fourth, performance metrics should not be the sole decision-making criterion determining implementation of a potential tool. A tool with low specificity may be acceptable if the purpose of risk triaging is to reduce the population of patients who may require additional diagnosis, care, or support, because even a relatively non-specific tool has the potential to identify those patients who do not require additional services. In any group of 100 clients initiating ART, for example, it is likely that at least 60, if not more, are at very low risk of disengagement from care in their first year [14,62]. Standard of care guidelines in many settings would require all one hundred of them to be assigned multiple adherence counselling sessions in the months after initiation. A triaging tool that could identify even half (30) of the true “low risk” patients would allow the facility to focus counselling resources more efficiently among those who are most likely to derive benefit from such an intervention. While not ideal, this would still represent a potentially valuable improvement in facility performance.

Finally, no risk triaging tool should be expected to perform consistently across all populations and settings. Setting matters when evaluating tool performance, even for a screening tool for TB risk, which relies on presumably objective symptom and laboratory indicators[63] and will be affected by TB prevalence rates in the setting the tool is implemented. For a behavioral risk tool, community and facility factors may be as important in predicting risk as individual characteristics. Travelling long distances to an ART clinic, for example, is frequently identified as a barrier to treatment adherence, and many programs offer community-based service delivery to address this barrier. In communities where stigma around HIV is high, however, individuals requiring HIV care may in fact prefer to travel further to facilities based in communities where they are less likely to be recognized by those to whom they have not disclosed their status. Local refinement of established risk triaging instruments may thus be desirable.

This review had a number of limitations. First, our search terms excluded all inpatient settings. This may have systematically excluded entire medical conditions that frequently use risk triaging approaches, such as disease due to the Ebola virus [64]. Second, while we included terms for performance test metrics such as sensitivity, specificity, and accuracy to increase the yield of studies reporting methodological results, we acknowledge that the terminology around test metrics is not universally standard and may have led to relevant studies being missed, particularly among older studies. Third, though we were unable to address this limitation through search methodology, we acknowledge that a systematic review will not observe tools that have not been described and published in academic literature. There may be some inherent reporting bias if results of tools that did not work were less likely to be published, but it is also possible that implementing partners and organizations have designed and implemented successful triaging tools and systems in primary health contexts but not evaluated or published these. Fourth, as discussed above, generalizability of the performance of any given risk triaging tool is uncertain. Tools that rely on clinical symptoms or test results may be applicable across multiple settings and populations, but tools that assess behavioural risks seem likely to be highly context-specific. Finally, in determining which papers to include in the analysis, the distinction between risk triaging and diagnosis was sometimes unclear and required author discretion; it is possible that some sources that others may regard as examples of risk triaging were categorized as diagnosis and excluded.

Conclusion

Despite the limitations listed above, our review highlights some important considerations for the implementation of risk triaging tools in the context of HIV retention. Many studies have achieved relatively impressive performance in identifying the risks they target within the data sets or small patient cohorts in which they were designed and validated. The lack of large-scale adoption and implementation of any of these instruments, however, points to serious gaps in previous efforts. Bringing implementers, policy makers, funders, and potentially clients themselves into the process of identifying needs, developing tools and intervention strategies, and assessing feasibility may be an essential component of improving health outcomes through risk stratification.

Ethics approval and consent to participate

This is a systematic review of published information and no direct contact with participants or access to participant data beyond what is reported in the published studies occurred.

Availability of data and materials

All data used in this paper are included in the cited manuscripts and publicly available in the cited sources.

Competing interests

The authors declare that they have no competing interests.

Funding

Funding for the study was provided the Bill & Melinda Gates Foundation through INV-031690 to Boston University. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Author contributions

MM, SR, and LS conceived of and designed the study. DF defined and ran the search. MM, LS, and MB identified and reviewed sources and extracted data. MM, LS, and MB analyzed the data and drafted the manuscript. All authors reviewed and edited the manuscript.

Supporting information

S1 Text. Review protocol S1 Table. Search strategy

S2 Table. Search terms-PubMed S3 Table. Prisma checklist.

S4 Table. Summary of bias using the Joanna Briggs Checklist for Diagnostic Test Accuracy