Barriers and facilitators to anti-retroviral therapy adherence among adolescents aged 10 to 19 years living with HIV in sub-Saharan Africa: A mixed-methods systematic review

Londiwe D. Hlophe, Jacques L. Tamuzi, Constance S. Shumba, Peter S. Nyasulu
doi: https://doi.org/10.1101/2022.10.07.22280823

Abstract

Background Human Immunodeficiency Virus (HIV) significantly affects adolescents globally, with the sub-Saharan Africa (SSA) reporting a high burden of the disease. HIV testing, treatment, and retention to care are low among adolescents. We conducted a mixed-method systematic review to assess anti-retroviral therapy (ART) adherence; barriers and facilitators to ART adherence and ART outcomes among adolescents living with HIV and on ART in sub-Saharan Africa.

Methods We conducted searches in four scientific databases for studies conducted between 2010 and March 2022 to identify relevant primary studies. Studies were screened against inclusion criteria and assessed for quality, and data was extracted. Meta-analysis of rates and odd ratios was used to plot the quantitative studies and meta-synthesis summarized the evidence from qualitative studies.

Results A total of 10 431 studies were identified and screened against the inclusion/ exclusion criteria. Sixty-six studies met the inclusion criteria (41 quantitative, 16 qualitative, and 9 mixed-methods study designs). Fifty-three thousand two hundred and seventeen (53 217) adolescents (52 319 in quantitative studies and 899 in qualitative studies) were included in the review. Thirteen support focused interventions for improved ART adherence were identified from quantitative studies. The plotted results from the meta-analysis found an ART adherence rate of 65% (95%CI 56-74), viral load suppression was 55% (95%CI 46-64), un-suppressed viral load rate of 41% (95%CI 32-50), and loss to follow up of 17% (95%CI 10-24) among adolescents. Meta-synthesis found six themes of barriers to ART (social, patient-based, economic, health system-based, therapy-based, and cultural barriers) in both the qualitative and quantitative studies, and three themes of facilitators to ART were also identified (social support, counselling, and ART education and secrecy or confidentiality) from qualitative studies.

Conclusion ART adherence remains low among adolescents in SSA despite multiple interventions implemented to improve ART adherence. The low adherence rate may hinder the attainment of the UNAIDS 2030 targets. Additionally, various barriers to ART adherence due to lack of support have been reported among this age group. However, interventions aimed at improving social support, educating, and counselling adolescents may improve and sustain ART adherence.

Systematic review registration PROSPERO CRD42021284891

1. Background

Adolescents are the most affected age group globally, with 1.75 million [1.16-2.3 million] of adolescents living with HIV [ALHIV) and 90% of these adolescents are found in the sub-Saharan Africa (SSA) region by July 2021 [1]. In 2020, adolescents accounted for 11% of new infections and 5% of AIDS-related deaths globally [2]. HIV testing is low among adolescents, especially in SSA countries. In a study conducted by Asaolu et al., 26.5% of adolescents aged 15 to 19-years had ever tested for HIV in sub-Saharan Africa, thus the proportion of those living with HIV and on treatment is low [3]. HIV treatment outcomes such as viral load suppression and AIDS-related death rate are also poor among adolescents due to low retention to care and poor treatment adherence rate [4, 5]. Accordingly, viral load suppression is low among adolescents when compared to adult population [6].

Viral load suppression (VLS) defined as viral load <50 RNA copies per ml of blood is associated with optimum ART adherence [7]. Optimum ART adherence is defined as correctly taking of medication doses above 95% [8]. Among adolescents, ART adherence is reported to be low and even lower in the SSA region. The sub-optimum adherence (less than 95% of correctly taking medication) is as a result of a number of factors namely; social, economic, therapy-related and health system-related and patient-related barriers [9].

Several interventions have been implemented to counteract these barriers and thus facilitate ART adherence among people living with HIV. These interventions are mainly focused in improving social support through the use of reminders and peers [10, 11]. Additionally, interventions aimed at improving facility-based barriers such as ART clinics and combination therapy have been implemented yielding positive ART treatment outcomes [12, 13]. World Health Organization guidelines have enabled the improved diagnosis and treatment initiation of people living with HIV. These include the 2010 guidelines which broadened the eligibility to ART to CD4 count of ≤350 cells/mm3 for all regardless of the clinical stage [14].

Targets aimed at collective effort to improve clinical outcomes have also been set such as the fast-track commitments by UNAIDS. These are the 90% reduction of new HIV cases by the year 2030 and the 61% reduction in HIV prevalence between 2010 and 2050 [15, 16]. Additionally, the fast-track 2030 targets aiming at 95% of those living with HIV knowing their HIV status, 95% of them on ART and 95% of those on ART with suppressed viral load. However, adolescents in the SSA region present poor outcomes as 43% are initiated on ART treatment with only 31% of those initiated on ART continue to access ART. Among those continually accessing ART, 30% are viral load suppressed [17, 18]. The COVID-19 pandemic has further negatively influenced the HIV outcomes globally [19, 20]. As a result of the lockdowns, there have been disruptions in access to HIV prevention and control services resulting in new infections and severe HIV treatment outcomes especially among adolescents from SSA. Within SSA, COVID 19 prevention and treatment options were limited with low vaccination coverage of 3% yet COVID-19 infection has been associated with severe outcomes among people living with HIV [1, 19–21].

Nevertheless, advances had been made pre-COVID-19 pandemic in the control and prevention of HIV yet adolescents still present poor HIV treatment outcomes.

There is scanty comprehensive adolescent-focused data on the barriers, facilitators of ART adherence, HIV treatment outcomes in SSA. Therefore, our review aimed at systematically and critically reviewing literature on ART adherence among adolescents living with HIV (ALHIV) in SSA by exploring evidence on: (a) the barriers and facilitators of ART adherence; (b) ART outcomes (ART adherence rate, loss to follow up, viral load suppression and non-suppression rates); (c) Types of interventions aimed at improved ART adherence among ALHIV from SSA.

2. Methods

The systematic review is registered with PROSPERO (Registration: CRD42021284891) [22]. The results are reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA statement) [23], and the Enhancing Transparency in Reporting the synthesis of Qualitative research (ENTREQ) recommendations [24].

2.1. Search strategies

A literature search including only studies conducted from 2010 to March 2022 was conducted. The following databases were used for this literature search; PubMed, Cochrane Review, Scopus on Excerpta Medica Database (Embase) and CINAHL. To identify studies, the following search terms were used: (Antiretroviral OR ART) AND (adherence Or Compliance) AND (Adolescents) AND (sub-Saharan Africa* OR sub-Saharan Africa region).

The inclusion criteria were research studies (qualitative, quantitative, and mixed method research studies) reporting on ART adherence outcomes among adolescents aged 10 to 19-years in sub-Saharan Africa. We only included English published articles. The following outcomes were included:

  • ART adherence rate: proportion of adolescents with optimum ART adherence among total adolescent in the study

  • ART adherence barriers: list and proportion of adolescents reporting either patient or individual, economic, social, health system-related, therapy-related and cultural barriers faced by participants in attaining optimum ART adherence [9,25,26].

  • Lost to follow up (LTFU): proportion adolescents lost to follow up.

  • Viral load (VL) suppression: proportion of adolescents acquiring VL suppression (VL <50 copies/ml) in the study.

  • Unsuppressed VL: proportion of adolescents with VL>50 copies/ml.

2.2. Study selection

Two investigators (LDH and JLT) independently assessed studies citations against the inclusion criteria. For citations where there was eligibility disagreement, PSN further independently resolved the disagreements. LDH and JLT further independently assessed included studies for eligibility through reading through the full text and disagreements were resolved through consensus between the two investigators. The review process is attached as Fig 1.

Fig 1:
Fig 1:

Flow diagram of studies including barriers and facilitators to anti-retroviral therapy adherence among HIV infected adolescents aged 10 to 19 years living in sub-Saharan Africa

2.3. Data extraction

We imported search outputs into Covidence library and removed duplicate references. Two data extraction forms were designed for quantitative and qualitative study designs. Mixed methods data extraction was conducted as separated in quantitative and qualitative designs. LDH and JLT independently extracted descriptive and analytical data from included studies (Table 1 and Table 2). Extracted data included the following: Author (s), title, year of publication, country, study design, study characteristics, participants demographic data, type of intervention, themes or outcomes, and final findings. Secondly, barriers to ART adherence were grouped into patient-related, therapy-related, health system-related, social, economic, and cultural factors [9] (Table 3 and Table 4). Lastly, facilitators to ART adherence were extracted from each included qualitative study (Table 5). Consensus of data extraction was obtained through discussions between JLT and LDH and JLT finally went through all citations and data extracted to ensure accuracy. In case of any discrepancy, PSN was consulted. Upon consensus on qualified studies, LDH and JLT entered relevant data for the excel data extraction forms from potentially eligible studies. Strengthening the reporting of epidemiological studies (STROBE) and the Consolidated Qualitative Study (COREQ) Reporting Standards for both quantitative and qualitative studies have guided the design of these types of forms [27].

View this table:
Table 1:

Overview of included qualitative studies

View this table:
Table 2:

Characteristics of included quantitative studies

View this table:
Table 3:

Barriers to anti-retroviral therapy adherence among adolescents aged 10 to 19 years living with HIV in sub-Saharan Africa (Quantitative studies)

View this table:
Table 4:

Barriers to ART adherence (Qualitative studies)

View this table:
Table 5:

Facilitators of ART adherence

2.4. Critical Appraisal

Study quality was independently assessed by LDH and JLT through using methodological quality criteria outlined for qualitative, quantitative and mixed method studies in the Mixed Method Appraisal Tool (MMAT) [28]. Qualitative studies were assessed using the following five domains: a) appropriateness of the approach in answering research question b) adequacy of data collection methods to answer research question c) if findings were derived from data presented d) interpretation of results sufficiently validated by data and e) coherence between data sources, collection, analysis, and interpretation. Quantitative studies were assessed based on either as randomized control trials, non-randomized or descriptive studies. For randomized control trials the following domains were used; a) randomization, b) comparability of the groups, c) completeness of outcome data, d) blinding of assessors and e) adherence to intervention by participants. Non-randomized trials were assessed against a) appropriateness of the measurements, b) representativeness of target population, c) completeness of outcome data, d) controlling for confounder at design, analysis, and e) if intervention was administered as intended. On the other hand, descriptive quantitative studies were assessed for: a) relevancy of sampling strategy, b) representativeness of target population, c) appropriateness of measurements, d) level of risk of bias and e) appropriateness of statistical analysis. Lastly, mixed method studies were assessed for: a) rationale for used the design, b) integration of the two different components, c) interpretation of the outputs of the integrated components, d) if discrepancy and inconsistences between the two components were addressed and e) if the two components adhered to quality criteria of each component. The final scores for the assessment were agreed through discussions between LDH and JLT. PSN subsequently resolved conflicts via discussion with the dual screeners.

2.5. Statistical analysis

Data analysis was done through parallel results convergent design where qualitative and quantitative findings were analyzed separately and integrated in the overall results. For quantitative analysis, a meta-analysis was conducted using Stata software version 16 and Open Meta analyst. Meta-analysis was performed to calculate the pooled estimates of the following outcomes: ART adherence, viral load suppression, unsuppressed viral load, and loss to follow up. The pooled effect size was calculated as rates and the odds ratios and their 95%CIs. As we expected high heterogeneity between studies, subgroup analysis was undertaken based on different ART barriers reported in the studies. In addition, we also performed meta-regression based on the adolescents age groups. A random effect meta-analysis was used to pool the estimates because of the variability of study designs and interventions. Heterogeneity was quantified using the I2 to identify study variation. Furthermore, publication bias was used to assess to the funnel plot symmetry and the Egger rank and Begg correlation tests were used to assess the quantitative publication bias. For statistical analysis, P-values of 0.05 were considered statistically significant. As Stata software version 16 reported the Log OR, we converted this to the OR.

Qualitative analysis was conducted through grouping key finding into the main themes according to both the barriers and facilitators to ART adherence as relevant to the study (Table 4 and 5). Similarly, barriers to ART adherence were analyzed for quantitative design studies narratively through grouping findings into main barriers to ART themes (Table 3). The themes used in this study were patient-related, social, therapy-related, health system-related, economic, and cultural barriers [9]. These themes were then subjected to a meta-synthesis to produce a single comprehensive set of synthesized findings.

3. Results

A total of 10 431 studies were found after running the search strategy in the different databases and 4744 duplicates were removed. The process followed the PRISMA guidelines as reported in Fig 1. Only 206 full-text studies met screening criteria and 66 studies retained and were included in the analysis: quantitative studies (n= 41), qualitative studies (n= 16), and mixed study (n = 9). Other studies were excluded because of wrong data collection period, wrong population group, non-disintegrated by age and one study was excluded because of its small sample size.

3.1. Characteristic of included studies

Out of the 66 studies included, 24 studies were included for the qualitative analysis of which 8 of these were the qualitative component of mixed-method studies as reported in Table 1. Most studies (7/24) included adolescents aged 10 to 19 years followed by 13 to 19-year-old (4/24) and 12 to 19-year-old (3/24). Majority (5/24) of the studies were conducted in South Africa followed by Kenya (4/24). The other studies originated from the following countries: Zambia (3), Uganda (3), Zimbabwe (2), Ghana (2), Rwanda (1), Malawi (1), Botswana (1), Tanzania (1), Mozambique (1). Of all the 24 studies, 10 used in-depth interviews to collect data while 7/24 used both focus group discussions and interviews for data collection. Semi-structured interviews were used by 4 studies while 3 used focused group discussions only.

Among the quantitative studies (Table 2), 48 studies were included in the meta-analysis of which 7 were the quantitative component of the mixed method studies. Majority of the studies (29) studies included adolescents aged 10 to 19 followed by 4 studies with adolescents aged 15 to 19 years. Twenty-one studies were cross-sectional in design while 19 were cohort studies. Nine studies were conducted in South Africa and another 9 in Zimbabwe followed by 7 studies conducted in Uganda. The other studies were conducted in the following countries: Kenya (5); Zambia (3); Ethiopia (3); Tanzania (3); Nigeria (3); Eswatini (1); Botswana (1); Namibia (1); Mozambique (1); Cameroon (1) and Malawi (1). The distribution of the studies including ART adherence among adolescent living with HIV and on ART is shown in Fig 2 below.

Fig 2:
Fig 2:

Map showing the studies included in the review

3.2. Quality assessment

The results of the quality assessment are shown in the S1 Table. For each domain, the score was “yes or no,” and a percentage of the “yeses” was used as the final score for each study. Among the qualitative studies, only one study scored less than 50% (29), two scored 60% [28, 29] and the remaining thirteen were scored 80 and above [25, 30, 39–41,31–38]. Among the quantitative studies, four were scored below 50% [42–45]. Seventeen scored 60% [46, 47, 56–62, 48–55] and nineteen scored 80% and above [56, 63, 72–80, 64–71]. Lastly, among the mixed studies, two studies scored 60% [81, 82] and the remaining seven scored 80% and above [83–89].

3.3. Interventions

A total of 13 interventions were identified in 19 of the quantitative studies included [42, 43, 77, 78, 81, 82, 90–93, 49, 50, 54, 55, 58, 63, 64, 74]. These were as follows: SMS reminders (65); family based financial asset [96]; enhanced counselling [43, 49, 50, 54, 74]; combination ART [92]; community adherence supporters [44]; universal test and treat [79]; peer mentoring [93]; teen clubs [57]; modified directly administered ART [60]; adolescents package of care [94]; integrated HIV and TB services [80]; mobile-based mental health [95] and peer support intervention and support groups [81, 82].

3.4. Meta-analysis

3.4.1. ART Adherence

ART adherence was reported in 22 studies from 11 countries in SSA (Fig 3). The overall ART adherence rate was 65% (95% CI 56-74; I2 99.01%) among adolescents on ART [44, 45, 65, 66, 69–71, 76, 77, 86, 87, 90, 47, 93, 95, 51, 53, 54, 56, 57, 62, 64]. The lowest ART adherence rate was reported in a study conducted in Tanzania while the highest rate was reported in a study conducted in Uganda (8% (95% CI 4-11)) and (93% (95% CI 79-100)) respectively [66, 87] (Fig 3 and S2 Fig).

Figure 3:
Figure 3:

Forest plot of ART adherence rate among adolescents aged from 10 to 19 years

Effects of ART adherence interventions showed that SMS reminders, community adherence supporters, teen clubs and modified directly administered ART were not statistically associated with ART adherence (OR 1.06 (95% CI 0.85-1.35, p= 0.62 I2 0.00%)) [44,57,60,65,77] (Fig 4). ART adherence was 1.2 (95% CI 0.75-1.93) higher among those reporting a SMS reminder compared to those not receiving a SMS reminder [65] (Fig 4). Among adolescents who received standard care and had access to a community adherence support intervention, ART adherence was 1.84 (95% CI 0.81-4.22) compared to adolescents only receiving standard care [44]. Modified directly administering of ART was associated with 1.77 (95% CI 0.66-4.66) ART adherence while teen clubs and a viral load >1000c/ml were negatively associated with ART adherence (OR 0.95 (95% CI 0.64-1.30)) and (OR 0.84 (95% CI 0.47-1.51)) respectively [55, 58, 75] (Fig 4).

Fig 4:
Fig 4:

Forest plot of ART adherence interventions among adolescents aged from 10 to 19 years

3.4.2. Barriers to ART adherence

Six categories of barriers to ART were reported in included studies. These were social, economic, health systems-related, therapy-related, patient-related and cultural barriers as adopted from previous studies [9]. Table 3 shows the barriers presented in the included studies.

ART adherence was high among studies reporting economic barriers (81% (95% CI 76.7-86.9)) followed by studies reporting therapy-related barriers (76 % (95% CI 68.5-83.5)) and health systems-related barriers (75.3% (95% CI 58.3-92.3; I2 98.7%)) (S2 Fig). ART adherence was low among studies reporting social barriers (45.2% (95% CI 19.9-70.4; I2 99.1%)) and patient reported barriers (45.2% (95% CI 37.7-52.8)). However, ART adherence was reduced in studies reporting combination of health systems-related and patient-related barriers (64% (95% CI 60.4-67.6)) [45, 54, 82, 87, 95],social and health systems-related barriers (52.5% (95% CI 28.9-76.1; I2 98.1%)); social and patient-related barriers (58.6% (95% CI 40.7-76.5)) [53, 57, 65, 66, 70, 77, 93] and economic and social barriers (63.4% (95% CI 59.7-67.2)) (S2 Fig). ART adherence was high among studies reporting a combination of social, health systems-related, patient-related and cultural barriers (83% (95% CI 79-87.1)); social, health systems-related and therapy-related barriers (72% (95% CI 769.4-74.9));social and therapy-related barriers (72.4% (95% CI 57.9-87; I2 84.5%)) [44, 47, 71, 76]; economic, social and therapy-related barriers (80% (95% CI 74.5-85.5)) [56, 86]; economic and social barriers (70.7% (95% CI 66.6-74.7)) and economic, health systems-related and patient-related barriers (70.3% (95% CI 66.3-74.3)) [62, 64, 90] (S3 Fig).

3.4.3. Viral load suppression

Fifteen studies were included in the analysis of the viral load suppression (VLS) rate among adolescent in SSA [45, 49, 73, 74, 77, 81, 85, 51, 52, 55, 56, 61, 65, 69, 71]. The overall viral load suppression rate was 55% (95% CI 46-64: I2 99.5%) (Fig 5). VLS was low among studies reporting social and therapy-related barriers (22% (95% CI 20-25; I2 98.7%)) and social barriers (48% (95% CI 31-61)) [51,67,73,75,81] (Fig 5). However, VLS was high among studies reporting therapy-related barriers (74% (95% CI 73-75; I2 22.2%)); economic, cultural and therapy related-barriers (65% (95% CI 59-72)) (Fig 5); health system-related barriers (64% (95% CI 57-71)); patient-related barriers (56% (95% CI 43-64; I2 68.1%)) and social, health system-related and therapy-related barriers (50% (95% CI 9%-94%)) [53,54,58,63,71,76,79,83,87] (Fig 5).

Fig 5:
Fig 5:

Forest Plot of viral load suppression and ART adherence barriers among adolescents aged from 10 to 19 years

Additionally, seven studies reported on interventions reporting ART adherence barriers and VLS [55, 58, 63, 75, 91, 92, 94]. VLS was not statistical associated with the barriers to ART (OR 1.25 (95% CI 1-1.55 p=0.05: I2 61.1%)). However, in a study conducted in Zimbabwe, health systems and therapy-related barriers were associated with over two-fold increase in viral load suppression (OR 2.48 (95%CI 1.51-4.10)) [77]. Economic (OR 1.04 (95%CI 0.81-1.32)) [96], social (OR 1.01 (95%CI 0.71-1.42; I2 0.00%)) [55, 91], health systems-related (OR 1.16 (95%CI 1.041.30)) [94]; patient-related (OR 1.77 (95%CI 0.61-5.05)) [60] and social and patient-related barriers (OR 1.35 (95%CI 085-2.16)) [65] were not associated with VLS (Fig 6).

Fig 6:
Fig 6:

Forest Plot of viral load suppression and intervention reporting ART adherence barriers among adolescents aged from 10 to 19 years

3.4.4. Unsuppressed viral load

A total of 11 studies assessed barriers to ART adherence and unsuppressed viral load rate among adolescents [45,51,52,63,67,71, 74–76,83,98)] (Fig 7). The overall unsuppressed viral load rate was 41% (95%CI 32-50; I2 96.5%) for included studies (Fig 7). The included studies reported the following barriers: social [49, 65, 73, 96]; therapy-related [63, 74]; patient-related [71, 83]; social and health system-related [76]; social and therapy-related [52] and lasty social, health system-related and therapy-related barriers [45] (Fig 7). Unsuppressed viral load rate was high in a single study reporting social and health system-related barriers (71% (95%CI 62-79)) [76] followed by studies reporting social barriers and patient-related barriers (43% (95% CI 29-57; I2 90.9%)) [49, 65, 73, 96]; (40% (95% CI 26-54; I2 60.7%)) [71, 83] respectively (Fig 7). Unsuppressed viral load was noted in studies reporting therapy-related barriers (26% (95% CI23-27; I2 0.02%)) [63, 74] as well as in a study reporting social and therapy-related barriers (34% (95% CI 24-45)) [45] (Fig 7).

Fig 7:
Fig 7:

Forest plot of unsuppressed viral load and ART adherence barriers among HIV infected adolescents aged from 10 to 19 years

Moreover, three studies reported interventions for adolescents associated with unsuppressed viral load [57,65,96] (Fig 8). The odds of VLS among adolescents receiving intervention were significant reduced compared to controls (OR 0.44 (95% CI 0.36-0.54: I2 0.00%)) (Fig 8).

Fig 8:
Fig 8:

Forest plot of ART adherence interventions and unsuppressed viral load among adolescents aged from 10 to 19 years

3.4.5. Loss to follow up

Among the 10 studies included, loss to follow up (LTFU) was reported to be 17% (95% CI 10-24; I2 99.1%) [45, 51, 60, 67, 68, 70, 78–80, 96] (Fig 9). LTFU was high (36% (95% CI 36-38)) among adolescents in a single study reporting health system and patient-related barriers [70] followed by studies reporting social barriers at 26% (95% CI 7-45; I2 96.9%) [79, 96] (Fig 9). Therapy-related barriers were associated with 18% (95% CI 5-31; I2 86.4%) [51, 78] LTFU while social and therapy-related was associated with 14% (95% CI 12-15) [82] (Fig 9). LTFU was less (6% (95% CI 5-9)) in a study conducted in South Africa reporting social and health system-related barriers followed by a study conducted in Zimbabwe reporting patient-related barriers with 9% (95% CI 5 -12) LTFU rate [67, 70] (Fig 9).

Fig 9:
Fig 9:

Forest of LTFU and ART adherence barriers among adolescents aged from 10 to 19 years

3.5. Meta regression

A meta-regression analysis was performed to examine the ART the relationship between ART adherence and age group. ART adherence statistically reduced in the age group of 10 to 16 and 10 to 18 with P = 0.002 and 0.005, respectively. In contrast, a borderline decreased of adherence was observed in the age group of 10-19 years, P = 0.056 (Table 6).

View this table:
Table 6:

Meta-regression output examining age group and ART adherence

3.6. Publication bias

The visual inspection of the funnel plots including ART adherence, viral load suppression and unsuppressed viral load rates were asymmetric (Fig 10,11 and 12). However, the Egger’s tests for small study effects and Begg’s test for ART adherence, viral load suppression and unsuppressed viral load rates were not statistically significant with the p-values of 0.2737; 0.3938, and 0.6147, respectively. Based on the above, publication bias may be considered less likely.

Fig 10:
Fig 10:

Funnel plot of studies reporting ART adherence rate among adolescents aged from 10 to 19 years

Fig 11:
Fig 11:

Funnel plot of studies reporting viral load suppression rate among adolescents aged from 10 to 19 years

Fig 12:
Fig 12:

Funnel plot of studies reporting unsuppressed viral load rate among adolescents aged from 10 to 19 years

3.7. Meta-synthesis

3.7.1. Barriers of ART adherence

Social barriers: Stigma and discrimination and lack of social support were the mostly identified social barriers to ART adherence among adolescents. Stigma and discrimination emanated from family members including parents and care-givers, community, friends, partners and school mates resulting to mental health effects such as depression and being suicidal [25, 28, 41, 87, 88, 93, 29–34, 38, 40]. While lack of support from caregivers and school mechanism resulted in adolescents being resentful and thus influenced their ART adherence patterns and resulting to hopelessness and suicidal thoughts too. Lack of social support was evident among adolescents with a non-parent caregiver [28, 30, 40, 88, 89]. Within the school mechanism, studies reported the clash in medicine taking time, classes, exams or tests period, living environment for those in boarding schools, busy schedule and the appointment time at the health facilities as barriers to ART adherence (29,33–36,40,42,88,89) .

Economic barriers

Poverty was reported to influence the ART adherence pattern among ALHIV. Poverty as explained in terms of lack of transport money to honor clinic schedules and lack of water and food to facilitate the correct uptake of medication resulted to adolescents missing clinic appointments and stopping medication (26,35,39,42,43,89,90).

Heath system-related barriers

Clinic or health facility factors were identified by adolescents as a barrier to ART adherence as a result of the location and distance to heath facility, waiting time at the facility, overcrowding, stock out, clinic protocols such as color coding, attitude of health-care workers, and rigid scheduling of appointments [27, 31, 33, 34, 36, 40, 86–89]. These factors influenced the ART adherence patterns of the adolescents as they were not supportive to the needs of ALHIV resulting in loss to follow up and defaulting of medication [27, 31, 86].

Therapy-related barriers

Regime, dosage frequency and side effects of medication were reported as the main barriers of ART adherence related to the treatment resulting to adolescents stopping medication [26,30,31, 34–36,41]. Additionally, adolescents complained about the packaging of the pills as it was easy to identify what it is due to the labelling and also the bottles were large and made lots of noise risking unintended disclosure of their status and treatment [38, 41].

Patient-related barriers

The attitude of adolescents, adolescents’ responsibility for treatment, and the ART knowledge were also a barrier to ART adherence as adolescents felt that being on ART posed restrictions on their life and caused them to miss opportunities their peers had [33,38,87]. Additionally, forgetfulness, tiredness, being busy, fatigue, being away from home were reported to also hinders adolescents’ ART adherence [26,30,34,36,40,42,43,87,89,91]. Adolescents stated that they also want to feel like others and just take pill holidays so they feel normal [91].Lastly, adolescents described how not knowing what the medication, misconceptions and myths about ART made them less interested in adhering to taking it [87, 89].

Cultural barriers

Religion and culture had a strong influence on ART adherence among adolescents as they stopped taking their medication because they were prayed for, and were told to believe they were healed or believed that traditional medicine will heal them [39, 89].

3.7.2. Facilitators of ART adherence

Adolescents identified three themes to be important in promoting adherence to ART. These were support, secrecy or confidentiality and counselling or education. The facilitators are presented in Table 5.

Social Support

Multiple studies reported that support system emanated from peers, family members, school, health facility, financial and personal or interpersonal support [43, 30–32, 35, 41, 85, 86, 93]. These were fundamentals in mitigating stigma, discrimination and isolation and thereby foster networks and connectiveness of adolescents and their peers while creating a support structure to encourage adolescents to adhere to their medication especially when they are fatigued [26,29, 31– 33,35,37,38, 40–43,84,87,89,91,95]. Lastly, financial support was reported to be critical in ensuring adolescents had access to and could adhere to medication through provision of transport to the clinic and food supply [31, 37, 87, 89].

Secrecy or confidentiality

Status disclosure and safe storage of medication were reported to positively influence ART adherence as they enabled adolescents to take charge of their lives through adhering to the treatment as they understand the benefits of the medication [25, 28, 29, 32, 34, 36, 37, 41, 84].

Counselling and ART education

This gives adolescents an understanding of their status and importance of adhering while allaying their anxiety and discarding myths and misconceptions [26,29,33,35,40,84,91,95].

3.8. Overall results

An ART adherence rate of 65% (95% CI 56-74) was reported among adolescents in the SSA region. Viral load suppression rate was 55% (95% CI 46-64) while an unsuppressed viral load rate of 41% (95% CI 32-50) was reported with 17% (95% CI 10-24) loss to follow up rate. Six themes of barriers to ART adherence were identified among adolescents in SSA. These were social barriers, economic barriers, patient-related barriers, health systems-related and cultural barriers. Lastly, three main facilitating factors were identified being social support, secrecy or confidentiality and counselling, and ART education. The findings also revealed that the more barriers reported, the lower the adherence to ART among adolescents. According to the meta-regression, ART adherence was lower in the age group of 10 to 18 years. Poor ART adherence was caused by a composite of social and patient-related barriers. Adolescents had lower viral load suppression and a high rate of un-suppression due to social, health system, and patient-related barriers. Finally, health system and patient-related barriers reported a high rate of follow-up loss among adolescents. In comparison to the poor impact of interventions (SMS reminders, community adherence supporters, and teen clubs) on ART adherence, ART adherence facilitators have proven to be effective and acceptable among HIV-infected adolescents.

4. Discussion

The systematic review identified 66 studies which were included in the analysis which expanded on prior review on the same topic; ART adherence among adolescents [97–99]. The findings of our review showed that ART adherence was 65% (95% CI 56-74) and viral suppression was 55% (95% CI 46-64) which was in consistence with the 62% viral suppression used as a proxy measure for ART adherence among adolescent globally as reported in a systematic review including 53 countries [102]. However, the findings reported lower VLS rates compared to 2012 findings where VLS was at 84% in low- and middle-income countries yet was in consistent with findings of a systematic review reporting 59% in SSA [101, 102] . Secondly, loss to follow up in our review was 17% (95% CI 10-24) which is lower to findings of a study conducted in 34 countries under the International epidemiology Database to Evaluate AIDS (IeDEA) where LTFU was reported to be 30% in 2018 [105]. Loss to follow up rate in this review indicate improvement when compared to studies conducted in 2007 in SSA where LTFU was 51% [106]. Additionally, our findings revealed that unsuppressed viral load is 41% in SSA countries which is higher than reported rates between 15.5% and 16.6 to 25.6% reported in Uganda and South Africa respectively [105, 106]. This however can be explained by findings of a systematic review conducted by Munyayi et al whereby different adolescent’s focused intervention to improve ART adherence among countries reported differing viral load outcomes [108]. Lastly, interventions for ART adherence in included studies were primarily support oriented similar to findings to studies conducted in middle- and low- income countries [13]. However, our findings show that interventions such as SMS reminders, community adherence supporters, and teen clubs did not improve ART adherence in adolescents. In contrast, ART adherence facilitators such as social support, secrecy or confidentiality, counselling, and ART education were effective and acceptable. This could be explained by the fact that ART adherence facilitators considered a holistic approach to improving ART adherence [109, 110]. Rather than focusing on a specific adolescent aspect of ART adherence, as described in the interventions included in the meta-analysis, improving ART adherence may necessitate a more comprehensive approach. This is further demonstrated in the composite, which includes social barriers where ART adherence is low. A comprehensive approach to overcoming social barriers has shown that social support can improve ART adherence in adolescents. Although quantitative studies incorporating these adherence facilitators are required in Sub-Saharan Africa, a comprehensive approach should be considered to improve ART adherence in adolescents aged 10 to 19 years.

Stigma and discrimination, and lack of social support were mostly experienced and perceived by adolescents and had a negative impact on the treatment uptake and adherence. Stigma and discrimination were interrelated with other barriers (health system-related, patient-related, therapy-related, cultural, and economic barriers) perceived and experienced by adolescents and has a major contribution in the treatment uptake and adherence. Even though treatment uptake is an ultimate responsibility of adolescents, the clinic factors such as distance to the clinic, waiting time, health-workers perceived attitudes had a negative impact on the adherence rate of adolescents. The health system-related factors combined with the economic factors were also interrelated as without financial assistance, access to clinic or health facility is limited resulting to compromised ART adherence. Cultural barriers such as religion and use of traditional medicine facilitated by social barriers and therapy-related especially ART knowledge, side effects, WHO stage, stigma, and discrimination negatively impacted on the uptake and adherence of ART. Our study is consistent with studies that showed that personal beliefs, stigma, trust or satisfaction with health care workers, health service-related factors, and social support are main predictors for ART adherence [97, 98, 109–111]. Nevertheless, findings of this review indicate that these barriers can be addressed through proving social support, ART education and counselling of ALHIV. The identified facilitators are consistent with findings of a systematic review including studies conducted between 2004 and 2016 in SSA, which identified 29 facilitators which were mainly aimed at supporting adolescents through peer, care-giver, health-care workers, counseling, religion and beliefs of adolescents [99].

Our systematic review has strengths and limitations. Firstly, a key strength is that we conducted a mixed method systematic review which allows for comprehensive analysis of ART adherence among adolescents. Secondly, the review builds and improves on previously conducted studies in the region which included less studies in their analysis. Thirdly, even though our review only included studies published in peer reviewed journals, being a mixed method systematic review justified the exclusion of unpublished literature. Fourthly, the use of a holistic quality appraisal process promoted the use of researchers’ skills and judgement to assess value of contribution of each study. Therefore, the inclusion of all studies despite the quality assessment score allowed for contribution of each study to the knowledge pool thus valuable insight to the study subject. However, as a limitation of the review, only studies conducted in English were included yet there could be substantial evidence in missed studies conducted in other languages. Secondly, our review only included studies conducted from 2010 excluding previously conducted studies. Our review may therefore be missing other content of the subject matter which might still be applicable in the recent era and could have changed the perspective of the findings.

5. Conclusion

Multiple barriers to ART adherence have been reported among adolescents. These barriers negatively affect the uptake of ART and thereby the adherence to ART among adolescents. Nonetheless, several interventions aimed at improving ART adherence have been implemented in SSA. These interventions are aimed at creating a crucial support system for adolescents to address barriers to ART among adolescents. These barriers linked to lack of support systems either from caregivers, peers, the health system, and school environments. Despite the interventions, adolescents still present low ART adherence, viral load suppression and high lost to follow up and high unsuppressed viral load. This could be an indication of lack of adolescents input during the design of the intervention. A comprehensive approach may be the best way to improve ART adherence in adolescents aged 10 to 19 years. There is however a need for a study aimed at solely studying the influence of these interventions on the ART outcome.

6. List of abbreviations

HIV: Human immunodeficiency virus; AIDS : Acquired immunodeficiency syndrome; ALHIV: adolescents living with HIV; ART: antiretroviral therapy; CD4 :cluster of differentiation 4; RNA: Ribonucleic acid; COVID-19: Corona virus disease 19; COREQ: Consolidated Qualitative Study; ENTREQ: Enhancing Transparency in Reporting the synthesis of Qualitative research; PROSPERO: Prospective register of systematic reviews; MMAT: Mixed Methods Appraisal tool; PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses; SSA: sub-Saharan Africa; STROBE: Strengthening the Reporting of Epidemiological Studies; UNAIDS: The Joint United Nations Programme on HIV/AIDS; WHO: World Health Organization; CINAHIL: Cumulative index of nursing and allied health literature; PubMed: Public/ publisher Medline; LTFU: Loss to follow up; VLS: Viral load suppression; SMS: Short messages services; TB: Tuberculosis; IeDEA: International epidemiology database to evaluate AIDS; VL: Viral load

7. Declaration

Ethics approval and consent to participate: Not applicable

Consent for publication: Not applicable

Availability of data and materials: All data generated or analyzed during this study are included in the published article and its supplementary information files

Competing interests: The authors declare that they have no competing interests

Funding: There was no funding for this study from any agency either commercial, not-for-profit, or public sector

Authors’ contributions: LDH conceptualized the study; LDH and JLT extracted and analyzed the data. LDH, JTL, CS and PSN interpreted the statistical output of the results; LDH drafted the manuscript; JLT, CS and PSN reviewed the manuscript drafts and provided critical intellectual input. All authors read and approved the final manuscript.

Data Availability

All relevant data are within the manuscript and its Supporting Information files

Supportive information captions

S1 Table. Result of quality assessment of included studies using the Mixed Method Appraisal Tool (MMAT)

S2 Figure. Forest plot of subgroup analysis of ART adherence in adolescents in Sub-Saharan African countries

S3 Figure. Forest plot of subgroup analysis of ART adherence and barriers in adolescents aged from 10 to 19 years

S4 Table. PRISMA 2020 Checklist

Acknowledgement

None

References

  1. 1.↵
    UNAIDS. Global HIV & AIDS statistics — Fact sheet | UNAIDS [Internet]. 2022 [cited 2021 Aug 24]. Available from: https://www.unaids.org/en/resources/fact-sheet
  2. 2.↵
    UNICEF. HIV and AIDS in Adolescents - UNICEF Data [Internet]. 2021 [cited 2021 Jan 12]. Available from: https://data.unicef.org/topic/adolescents/hiv-aids/
  3. 3.↵
    Asaolu IO, Gunn JK, Center KE, Koss MP, Iwelunmor JI, Ehiri JE. Predictors of HIV Testing among Youth in Sub-Saharan Africa: A Cross-Sectional Study. PLoS One. 2016 Oct 1;11(10):e0164052.
    OpenUrlPubMed
  4. 4.↵
    Adejumo OA, Malee KM, Ryscavage P, Hunter SJ, Taiwo BO. Contemporary issues on the epidemiology and antiretroviral adherence of HIV-infected adolescents in sub-Saharan Africa: A narrative review. J Int AIDS Soc. 2015;18(1):20049.
    OpenUrlCrossRefPubMed
  5. 5.↵
    Amour MA, Shayo GA, Matee MM, Machumi L, Rugarabamu A, Aris EA, et al. Predictors of mortality among adolescents and young adults living with HIV on antiretroviral therapy in Dar es Salaam, Tanzania: a retrospective cohort study. J Int AIDS Soc. 2022 Feb 1;25(2):25886.
    OpenUrl
  6. 6.↵
    Han WM, Law MG, Egger M, Wools-Kaloustian K, Moore R, McGowan C, et al. Global estimates of viral suppression in children and adolescents and adults on antiretroviral therapy adjusted for missing viral load measurements: a multiregional, retrospective cohort study in 31 countries. Lancet HIV. 2021 Dec 1;8(12):e766–75.
    OpenUrlPubMed
  7. 7.↵
    WHO. WHO publishes new Consolidated HIV guidelines for prevention, treatment, service delivery & monitoring [Internet]. 2021 [cited 2022 Aug 10]. Available from: https://www.who.int/news/item/16-07-2021-who-publishes-new-consolidated-hiv-guidelines-for-prevention-treatment-service-delivery-monitoring
  8. 8.↵
    Achappa B, Madi D, Bhaskaran U, Ramapuram JT, Rao S, Mahalingam S. Adherence to Antiretroviral Therapy Among People Living with HIV. N Am J Med Sci. 2013 Mar;5(3):220.
    OpenUrlCrossRefPubMed
  9. 9.↵
    Azia IN, Mukumbang FC, Van Wyk B. Barriers to adherence to antiretroviral treatment in a regional hospital in Vredenburg, Western Cape, South Africa. South Afr J HIV Med. 2016 Sep 30;17(1):476.
    OpenUrl
  10. 10.↵
    Okonji EF, Mukumbang FC, Orth Z, Vickerman-Delport SA, Van Wyk B. Psychosocial support interventions for improved adherence and retention in ART care for young people living with HIV (10-24 years): a scoping review. BMC Public Health. 2020 Dec 1;20(1):1841.
    OpenUrl
  11. 11.↵
    Mbuagbaw L, Sivaramalingam B, Navarro T, Hobson N, Keepanasseril A, Wilczynski NJ, et al. Interventions for Enhancing Adherence to Antiretroviral Therapy (ART): A Systematic Review of High Quality Studies. AIDS Patient Care STDS. 2015 May 5;29(5):248.
    OpenUrlCrossRefPubMed
  12. 12.↵
    Casale M, Carlqvist A, Cluver L. Recent Interventions to Improve Retention in HIV Care and Adherence to Antiretroviral Treatment among Adolescents and Youth: A Systematic Review. AIDS Patient Care STDS. 2019 Jun;33(6):237–52.
    OpenUrlCrossRefPubMed
  13. 13.↵
    Reif LK, Abrams EJ, Arpadi S, Elul B, McNairy ML, Fitzgerald DW, et al. Interventions to Improve Antiretroviral Therapy Adherence Among Adolescents and Youth in Low- and Middle-Income Countries: A Systematic Review 2015–2019. AIDS Behav. 2020;(0123456789):1–14.
    OpenUrl
  14. 14.↵
    WHO. Antiretroviral therapy for HIV infection in adults and adolescents; Recommendations for a public health approach 2010 revision [Internet]. 2010 [cited 2022 Jun 7]. p. 1–156. Available from: http://apps.who.int/iris/bitstream/handle/10665/44379/9789241599764_eng.pdf;jsessionid=932287F8A5264774C8ED835AA641BC3F?sequence=1
  15. 15.↵
    Khalifa A, Stover J, Mahy M, Idele P, Porth T, Lwamba C. Demographic change and HIV epidemic projections to 2050 for adolescents and young people aged 15-24. Glob Health Action. 2019;00.
  16. 16.↵
    UNAIDS. Fast-track commitments to end AIDS by 2030 | UNAIDS [Internet]. Joint United Nations Programme on HIV and AIDS. 2016 [cited 2022 Jun 6]. p. 1. Available from: https://www.unaids.org/sites/default/files/media_asset/fast-track-commitments_en.pdf
  17. 17.↵
    Iyun V, Technau KG, Vinikoor M, Yotebieng M, Vreeman R, Abuogi L, et al. Variations in the characteristics and outcomes of children living with HIV following universal ART in sub-Saharan Africa (2006-17): a retrospective cohort study. Lancet HIV. 2021 Jun;8(6):e353–62.
    OpenUrl
  18. 18.↵
    Vreeman RC, Rakhmanina NY, Nyandiko WM, Puthanakit T, Kantor R. Are we there yet? 40 years of successes and challenges for children and adolescents living with HIV. J Int AIDS Soc. 2021 Jun 1;24(6):e25759.
    OpenUrl
  19. 19.↵
    Ssentongo P, Heilbrunn ES, Ssentongo AE, Advani S, Chinchilli VM, Nunez JJ, et al. Epidemiology and outcomes of COVID-19 in HIV-infected individuals: a systematic review and meta-analysis. Scientific Reports 2021 11:1. 2021 Mar 18;11(1):1–12.
    OpenUrl
  20. 20.↵
    Barbera LK, Kamis KF, Rowan SE, Davis AJ, Shehata S, Carlson JJ, et al. HIV and COVID-19: review of clinical course and outcomes. HIV Res Clin Pract. 2021;22(4):102–18.
    OpenUrl
  21. 21.↵
    Chenneville T, Gabbidon K, Hanson P, Holyfield C. The Impact of COVID-19 on HIV Treatment and Research: A Call to Action.
  22. 22.↵
    Hlophe LD, Tamuzi JL, Shumba C, Nyasulu PS. Barriers to anti-retroviral therapy adherence among adolescents aged 10 to 19 years living with HIV in sub-Saharan Africa: A mixed-methods systematic review protocol. Vousden G, editor. PLoS One [Internet]. 2022 Sep 30 [cited 2022 Oct 3];17(9):e0273435. Available from: https://dx.plos.org/10.1371/journal.pone.0273435
    OpenUrl
  23. 23.↵
    Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021 Mar 29;372.
  24. 24.↵
    Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Medical Research Methodology BioMed Central; Nov 27, 2012 p. 1–8.
    OpenUrl
  25. 25.↵
    Xu L, Munir K, Kanabkaew C, Le Coeur S. Factors influencing antiretroviral treatment suboptimal adherence among perinatally hivinfected adolescents in Thailand. PLoS One. 2017;12(2):1–18.
    OpenUrlCrossRefPubMed
  26. 26.↵
    Ankrah DNAA, Koster ES, Mantel-Teeuwisse AK, Arhinful DK, Agyepong IA, Lartey M. Facilitators and barriers to antiretroviral therapy adherence among adolescents in Ghana. Patient Prefer Adherence. 2016;10:329–37.
    OpenUrl
  27. 27.↵
    Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care [Internet]. 2007 Dec 1 [cited 2022 Sep 19];19(6):349–57. Available from: https://academic.oup.com/intqhc/article/19/6/349/1791966
    OpenUrl
  28. 28.↵
    Quan Nha HONGa, Pierre PLUYEa,, Sergi FÀBREGUESb, Gillian BARTLETTa FBoardman, Margaret CARGOd, Pierre DAGENAISe, Marie-Pierre GAGNONf, Frances GRIFFITHSc BNicolau, Alicia O’CATHAINg, Marie-Claude ROUSSEAUh & IVedel. MIXED METHODS APPRAISAL TOOL (MMAT) VERSION 2018 User guide [Internet]. 2018 [cited 2022 Jun 20]. Available from: http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf
  29. 29.↵
    Ritchwood TD, Ba A, Ingram L, Atujuna M, Marcus R, Ntlapo N, et al. Community perspectives of South African adolescents’ experiences seeking treatment at local HIV clinics and how such clinics may influence engagement in the HIV treatment cascade: a qualitative study. AIDS Care. 2020 Jan;32(1):83–8.
    OpenUrl
  30. 30.↵
    Kunapareddy CJ, Nyandiko W, Inui T, Ayaya S, Marrero DG, Vreeman R. A qualitative assessment of barriers to antiretroviral therapy adherence among adolescents in Western Kenya. J HIV AIDS Soc Serv. 2014;13(4):383–401.
    OpenUrl
  31. 31.↵
    Orth Z, van Wyk B. A Facility-based Family Support Intervention to Improve Treatment Outcomes for Adolescents on Antiretroviral Therapy in the Cape Metropole, South Africa. J Int Assoc Provid AIDS Care. 2021;20:1–11.
    OpenUrl
  32. 32.↵
    Enane LA, Apondi E, Omollo M, Toromo JJ, Bakari S, Aluoch J, et al. “I just keep quiet about it and act as if everything is alright” – The cascade from trauma to disengagement among adolescents living with HIV in western Kenya. J Int AIDS Soc. 2021;24(4):1–11.
    OpenUrlPubMed
  33. 33.↵
    Jimu C, Govender K, Kanyemba R, Ngbesso MJO. Experiences of intimate relationships, stigma, social support and treatment adherence among HIV-positive adolescents in Chiredzi district, Zimbabwe. African Journal of AIDS Research. 2021;20(3):214–23.
    OpenUrl
  34. 34.↵
    MacCarthy S, Saya U, Samba C, Birungi J, Okoboi S, Linnemayr S. “how am i going to live?”: Exploring barriers to ART adherence among adolescents and young adults living with HIV in Uganda. BMC Public Health. 2018;18(1):1–11.
    OpenUrlCrossRefPubMed
  35. 35.↵
    Madiba S, Josiah U. Perceived stigma and fear of unintended disclosure are barriers in medication adherence in adolescents with perinatal HIV in Botswana: A qualitative study. Biomed Res Int. 2019;2019.
  36. 36.↵
    Mutwa PR, Van Nuil JI, Asiimwe-Kateera B, Kestelyn E, Vyankandondera J, Pool R, et al. Living situation affects adherence to combination antiretroviral therapy in HIV-infected adolescents in Rwanda: a qualitative study. PLoS One. 2013;8(4):e60073.
    OpenUrlCrossRefPubMed
  37. 37.↵
    Rencken CA, Harrison AD, Mtukushe B, Bergam S, Pather A, Sher R, et al. “Those People Motivate and Inspire Me to Take My Treatment.” Peer Support for Adolescents Living With HIV in Cape Town, South Africa. J Int Assoc Provid AIDS Care. 2021;20:23259582211000524.
    OpenUrl
  38. 38.↵
    Burns R, Magalasi D, Blasco P, Szumilin E, Pasquier E, Schramm B, et al. “We give them threatening advice…”: expectations of adherence to antiretroviral therapy and their consequences among adolescents living with HIV in rural Malawi. J Int AIDS Soc. 2020 Mar;23(3):e25459.
    OpenUrl
  39. 39.↵
    Appiah SCY, Kroidl I, Hoelscher M, Ivanova O, Dapaah JM. A Phenomenological Account of HIV Disclosure Experiences of Children and Adolescents from Northern and Southern Ghana. Int J Environ Res Public Health. 2019 Feb;16(4).
  40. 40.↵
    Apondi E, Wachira J, Ayikukwei R, Kafu C, Onyango J, Omollo M, et al. Barriers to ART adherence among school students living with HIV in Kenya. Afr J AIDS Res. 2021 Oct;20(3):232–7.
    OpenUrl
  41. 41.↵
    Denison JA, Banda H, Dennis AC, Packer C, Nyambe N, Stalter RM, et al. The sky is the limit: Adhering to antiretroviral therapy and HIV self-management from the perspectives of adolescents living with HIV and their adult caregivers. J Int AIDS Soc. 2015;18(1):1–6.
    OpenUrl
  42. 42.↵
    van Wyk BE, Davids LAAC. Challenges to HIV treatment adherence amongst adolescents in a low socio-economic setting in Cape Town. South Afr J HIV Med. 2019;20(1):1–7.
    OpenUrlCrossRefPubMed
  43. 43.↵
    Kihumuro RB, Muganzi DJ, Wandira EG, Alinaiswe R, Nanyunja JJ, Kugumisiriza R, et al. Influence of boarding secondary school environment on HIV positive students in South Western Uganda. BMC Public Health. 2021;21(1):1–11.
    OpenUrlCrossRefPubMed
  44. 44.↵
    Willis N, Milanzi A, Mawodzeke M, Dziwa C, Armstrong A, Yekeye I, et al. Effectiveness of community adolescent treatment supporters (CATS) interventions in improving linkage and retention in care, adherence to ART and psychosocial well-being: A randomised trial among adolescents living with HIV in rural Zimbabwe. BMC Public Health. 2019;19(1):1–9.
    OpenUrlCrossRefPubMed
  45. 45.↵
    Bvochora T, Satyanarayana S, Takarinda KC, Bara H, Chonzi P, Komtenza B, et al. Enhanced adherence counselling and viral load suppression in HIV seropositive patients with an initial high viral load in Harare, Zimbabwe: Operational issues. PLoS One. 2019;14(2:1–13.
    OpenUrlCrossRefPubMed
  46. 46.↵
    Gitahi-Kamau N, Wahome S, Bukusi EA, Ngure K. Determinants of antiretroviral therapy adherence among older adolescents living with HIV in Kenya during the transition to adult care; An observational study. J AIDS HIV Res. 2020 Jul;12(2):24–33.
    OpenUrl
  47. 47.↵
    Meloni ST, Agaba P, Chang CA, Yiltok E, Oguche S, Ejeliogu E, et al. Longitudinal evaluation of adherence, retention, and transition patterns of adolescents living with HIV in Nigeria. PLoS One. 2020;15(7):e0236801.
    OpenUrl
  48. 48.↵
    Bermudez LG, Ssewamala FM, Neilands TB, Lu L, Jennings L, Nakigozi G, et al. Does Economic Strengthening Improve Viral Suppression Among Adolescents Living with HIV? Results From a Cluster Randomized Trial in Uganda. AIDS and Behavior 2018 22:11. 2018 May 30;22(11):3763–72.
    OpenUrl
  49. 49.↵
    Bongfen MC, Torpey K, Ganle J, Ankomah A. Level of adherence and associated factors among HIV-positive adolescents on antiretroviral therapy in Cameroon. Afr J AIDS Res. 2020 Dec;19(4):269–75.
    OpenUrl
  50. 50.↵
    Haghighat R, Toska E, Bungane N, Cluver L. The HIV care cascade for adolescents initiated on antiretroviral therapy in a health district of South Africa: a retrospective cohort study. BMC Infect Dis. 2021 Jan;21(1):60.
    OpenUrlCrossRef
  51. 51.↵
    Moyo S, Ncube RT, Shewade HD, Ngwenya S, Ndebele W, Takarinda KC, et al. Children and adolescents on anti-retroviral therapy in Bulawayo, Zimbabwe: How many are virally suppressed by month six? F1000Res. 2020;9(May):1–10.
    OpenUrl
  52. 52.↵
    Jobanputra K, Parker LA, Azih C, Okello V, Maphalala G, Kershberger B, et al. Factors associated with virological failure and suppression after enhanced adherence counselling, in children, adolescents and adults on antiretroviral therapy for HIV in Swaziland. PLoS One. 2015;10(2):e0116144.
    OpenUrlCrossRefPubMed
  53. 53.↵
    Tanyi WN, Gachuno O, Odero T, Farquhar C, Kimosop D, Mayi A. Factors affecting adherence to antiretroviral therapy among children and adolescents living with HIV in the Mbita Sub-County Hospital, Homa Bay-Kenya. Afr Health Sci. 2021 May;21(Suppl):18–24.
    OpenUrl
  54. 54.↵
    Desta AA, Wubayehu Woldearegay T, Berhe AA, Futwi N, Gebremedhn Gebru G, Godefay H, et al. Immunological recovery, failure and factors associated with CD-4 T-cells progression over time, among adolescents and adults living with HIV on Antiretroviral Therapy in Northern Ethiopia : A retrospective cross sectional study. PLoS One. 2019;14(12):1–18.
    OpenUrlCrossRefPubMed
  55. 55.↵
    Zhou S, Cluver L, Shenderovich Y, Toska E. Uncovering ART adherence inconsistencies: An assessment of sustained adherence among adolescents in South Africa. J Int AIDS Soc. 2021 Oct;24(10):e25832.
    OpenUrl
  56. 56.↵
    Gross R, Bandason T, Langhaug L, Mujuru H, Lowenthal E, Ferrand R. Factors associated with self-reported adherence among adolescents on antiretroviral therapy in Zimbabwe. AIDS Care - Psychological and Socio-Medical Aspects of AIDS/HIV. 2015;27(3):322–6.
    OpenUrl
  57. 57.↵
    Munyayi FK, van Wyk BE. The comparison of teen clubs vs. Standard care on treatment outcomes for adolescents on antiretroviral therapy in Windhoek, Namibia. AIDS Res Treat. 2020;2020:1–9.
    OpenUrlPubMed
  58. 58.↵
    Natukunda J, Kirabira P, Ong KIC, Shibanuma A, Jimba M. Virologic failure in HIV-positive adolescents with perfect adherence in Uganda: a cross-sectional study. Trop Med Health. 2019;47(1):1–10.
    OpenUrl
  59. 59.
    Ndiaye M, Nyasulu P, Nguyen H, Lowenthal ED, Gross R, Mills EJ, et al. Risk factors for suboptimal antiretroviral therapy adherence in HIV-infected adolescents in Gaborone, Botswana: a pilot cross-sectional study. Patient Prefer Adherence. 2013;7:891.
    OpenUrl
  60. 60.↵
    Chawana TD, Katzenstein D, Nathoo K, Ngara B, Nhachi CFB. Evaluating an enhanced adherence intervention among HIV positive adolescents failing atazanavir/ritonavir-based second line antiretroviral treatment at a public health clinic. J AIDS HIV Res. 2017 Jan;9(1):17–30.
    OpenUrl
  61. 61.↵
    Mburu M, Guzé MA, Ong’wen P, Okoko N, Moghadassi M, Cohen CR, et al. Evaluating the effectiveness of the HIV adolescent package of care (APOC) training on viral load suppression in Kenya. Public Health. 2019 Aug;173:146–9.
    OpenUrlPubMed
  62. 62.↵
    Vogt F, Rehman AM, Kranzer K, Nyathi M, Van Griensven J, Dixon M, et al. Relationship Between Time to Initiation of Antiretroviral Therapy and Treatment Outcomes: A Cohort Analysis of ART Eligible Adolescents in Zimbabwe. J Acquir Immune Defic Syndr. 2017 Apr;74(4):390–8.
    OpenUrl
  63. 63.↵
    Okonji EF, van Wyk B, Mukumbang FC, Hughes GD. Determinants of viral suppression among adolescents on antiretroviral treatment in Ehlanzeni district, South Africa: a cross-sectional analysis. AIDS Res Ther. 2021 Oct;18(1:66.
    OpenUrl
  64. 64.↵
    Brathwaite R, Ssewamala FM, Neilands TB, Okumu M, Mutumba M, Damulira C, et al. Predicting the individualized risk of poor adherence to ART medication among adolescents living with HIV in Uganda: the Suubi+Adherence study. J Int AIDS Soc. 2021 Jun;24(6):e25756.
    OpenUrl
  65. 65.↵
    Abiodun O, Ladi-Akinyemi B, Olu-Abiodun O, Sotunsa J, Bamidele F, Adepoju A, et al. A Single-Blind, Parallel Design RCT to Assess the Effectiveness of SMS Reminders in Improving ART Adherence Among Adolescents Living with HIV (STARTA Trial). Journal of Adolescent Health. 2021 Apr;68(4):728–36.
    OpenUrl
  66. 66.↵
    Bermudez LG, Jennings L, Ssewamala FM, Nabunya P, Mellins C, McKay M. Equity in adherence to antiretroviral therapy among economically vulnerable adolescents living with HIV in Uganda. AIDS Care - Psychological and Socio-Medical Aspects of AIDS/HIV. 2016;28(sup2):83–91.
    OpenUrl
  67. 67.↵
    Bitwale NZ, Mnzava DP, Kimaro FD, Jacob T, Mpondo BCTT, Jumanne S. Prevalence and Factors Associated with Virological Treatment Failure among Children and Adolescents on Antiretroviral Therapy Attending HIV/AIDS Care and Treatment Clinics in Dodoma Municipality, Central Tanzania. J Pediatric Infect Dis Soc. 2021 Mar;10(2):131–40.
    OpenUrl
  68. 68.↵
    Bulali RE, Kibusi SM, Mpondo BCT. Factors Associated with HIV Status Disclosure and Its Effect on Treatment Adherence and Quality of Life among Children 6-17 Years on Antiretroviral Therapy in Southern Highlands Zone, Tanzania: Unmatched Case Control Study. Int J Pediatr. 2018;2018:8058291.
    OpenUrl
  69. 69.↵
    Bygrave H, Mtangirwa J, Ncube K, Ford N, Kranzer K, Munyaradzi D. Antiretroviral therapy outcomes among adolescents and youth in rural Zimbabwe. PLoS One. 2012;7(12):e52856.
    OpenUrlCrossRefPubMed
  70. 70.↵
    Chaudhury S, Hertzmark E, Muya A, Sando D, Ulenga N, Machumi L, et al. Equity of child and adolescent treatment, continuity of care and mortality, according to age and gender among enrollees in a large HIV programme in Tanzania. J Int AIDS Soc. 2018;21:e25070.
    OpenUrl
  71. 71.↵
    Crowley T, van der Merwe A, Kidd M, Skinner D. Adolescent human immunodeficiency virus self-management: Associations with treatment adherence, viral suppression, sexual risk behaviours and health-related quality of life. South Afr J HIV Med. 2020;21(1):1–11.
    OpenUrl
  72. 72.↵
    Cluver L, Shenderovich Y, Toska E, Rudgard WE, Zhou S, Orkin M, et al. Clinic and care: associations with adolescent antiretroviral therapy adherence in a prospective cohort in South Africa. AIDS. 2021 Jul;35(8):1263–71.
    OpenUrl
  73. 73.↵
    Cluver L, Pantelic M, Toska E, Orkin M, Casale M, Bungane N, et al. STACKing the odds for adolescent survival: health service factors associated with full retention in care and adherence amongst adolescents living with HIV in South Africa. J Int AIDS Soc. 2018 Sep;21(9):e25176.
    OpenUrlPubMed
  74. 74.↵
    Wakooko P, Gavamukulya Y, Wandabwa JN. Viral load Suppression and Associated Factors among HIV Patients on Antiretroviral Treatment in Bulambuli District, Eastern Uganda: A Retrospective Cohort Study. Infect Dis. 2020;13:1178633720970632.
    OpenUrl
  75. 75.↵
    Umar E, Levy JA, Bailey RC, Donenberg G, Hershow RC, Mackesy-Amiti ME. Virological non-suppression and its correlates among adolescents and young people living with HIV in Southern Malawi. AIDS Behav. 2019;23(2):513–22.
    OpenUrl
  76. 76.↵
    Nasuuna E, Kigozi J, Babirye L, Muganzi A, Sewankambo NK, Nakanjako D. Low HIV viral suppression rates following the intensive adherence counseling (IAC) program for children and adolescents with viral failure in public health facilities in Uganda. BMC Public Health. 2018 Aug;18(1):1048.
    OpenUrlPubMed
  77. 77.↵
    Sithole Z, Mbizvo E, Chonzi P, Mungati M, Juru TP, Shambira G, et al. Virological failure among adolescents on ART, Harare City, 2017-a case-control study. BMC Infect Dis. 2018;18(1):1–8.
    OpenUrlCrossRef
  78. 78.↵
    Firdu N, Enquselassie F, Jerene D. HIV-infected adolescents have low adherence to antiretroviral therapy: A cross-sectional study in Addis Ababa, Ethiopia. Pan African Medical Journal. 2017;27(1):1–11.
    OpenUrl
  79. 79.↵
    Kabogo J, Muniu E, Wamunyokoli F, Musoke R, Songok E. Evidence of reduced treatment adherence among HIV infected paediatric and adolescent populations in Nairobi at the onset of the UNAIDS Universal Test and Treat Program. BMC Res Notes. 2018;11(1):1–7.
    OpenUrlCrossRefPubMed
  80. 80.↵
    Matyanga CMJ, Takarinda KC, Owiti P, Mutasa-Apollo T, Mugurungi O, Buruwe L, et al. Outcomes of antiretroviral therapy among younger versus older adolescents and adults in an urban clinic, Zimbabwe. Public Health Action. 2016 Jun;6(2):97–104.
    OpenUrl
  81. 81.↵
    Van Wyk BE, Kriel E, Mukumbang FC. Two-year viral load suppression among adolescents receiving antiretroviral therapy in the Cape Metropole, South Africa, 2013 - 2015: A retrospective cohort analysis. South African Medical Journal. 2020 Nov;110(12):1213–7.
    OpenUrl
  82. 82.↵
    Jerene D, Abebe W, Taye K, Ruff A, Hallstrom I. Adolescents living with HIV are at higher risk of death and loss to follow up from care: Analysis of cohort data from eight health facilities in Ethiopia. PLoS One. 2019;14(10):1–15.
    OpenUrlCrossRefPubMed
  83. 83.↵
    Dulli L, Ridgeway K, Packer C, Plourde KF, Mumuni T, Idaboh T, et al. An online support group intervention for adolescents living with HIV in Nigeria: A pre-post test study. JMIR Public Health Surveill. 2018 Oct 1;4(4):1–17.
    OpenUrl
  84. 84.↵
    Stangl AL, Mwale M, Sebany M, Mackworth-Young CR, Chiiya C, Chonta M, et al. Feasibility, Acceptability and Preliminary Efficacy of Tikambisane (’Let’s Talk to Each Other’): A Pilot Support Group Intervention for Adolescent Girls Living With HIV in Zambia. J Int Assoc Provid AIDS Care. 2021;20:23259582211024772.
    OpenUrl
  85. 85.↵
    Chory A, Callen G, Nyandiko W, Njoroge T, Ashimosi C, Aluoch J, et al. A Pilot Study of a Mobile Intervention to Support Mental Health and Adherence Among Adolescents Living with HIV in Western Kenya. AIDS Behav. 2022 Jul 22;26(1):232–42.
    OpenUrl
  86. 86.↵
    Cluver LD, Hodes RJ, Toska E, Kidia KK, Orkin FM, Sherr L, et al. “HIV is like a tsotsi. ARVs are your guns”: Associations between HIV-disclosure and adherence to antiretroviral treatment among adolescents in South Africa. Aids. 2015;29(April):S57–65.
    OpenUrl
  87. 87.↵
    Falcão J, Zerbe A, Mellins CA, Mantell J, Brittain K, Kapogiannis B, et al. The secret life of young adolescents living with HIV in northern Mozambique - a mixed methods study. BMC Public Health. 2021 Sep;21(1):1650.
    OpenUrl
  88. 88.↵
    Mesic A, Halim N, MacLeod W, Haker C, Mwansa M, Biemba G. Facilitators and barriers to adherence to antiretroviral therapy and retention in care among adolescents living with HIV/AIDS in Zambia: A mixed methods study. AIDS Behav. 2019;23(9):2618–28.
    OpenUrl
  89. 89.↵
    Nabukeera-Barungi N, Elyanu P, Asire B, Katureebe C, Lukabwe I, Namusoke E, et al. Adherence to antiretroviral therapy and retention in care for adolescents living with HIV from 10 districts in Uganda. BMC Infect Dis. 2015;15(1):1–10.
    OpenUrlCrossRefPubMed
  90. 90.↵
    Mavhu W, Berwick J, Chirawu P, Makamba M, Copas A, Dirawo J, et al. Enhancing psychosocial support for HIV positive adolescents in Harare, Zimbabwe. PLoS One. 2013;8(7):e70254.
    OpenUrl
  91. 91.↵
    Nyogea D, Mtenga S, Henning L, Franzeck FC, Glass TR, Letang E, et al. Determinants of antiretroviral adherence among HIV positive children and teenagers in rural Tanzania: a mixed methods study. BMC Infect Dis. 2015;15(1):1–13.
    OpenUrlCrossRefPubMed
  92. 92.↵
    Natukunda HPM, Cluver LD, Toska E, Musiime V, Yakubovich AR. Beyond clinical trials: Cross-sectional associations of combination antiretroviral therapy with reports of multiple symptoms and non-adherence among adolescents in South Africa. S Afr Med J. 2017 Oct;107(11):965–75.
    OpenUrl
  93. 93.↵
    Denison JA, Burke VM, Miti S, Nonyane BASS, Frimpong C, Merrill KG, et al. Project YES! Youth Engaging for Success: A randomized controlled trial assessing the impact of a clinic-based peer mentoring program on viral suppression, adherence and internalized stigma among HIV-positive youth (15-24 years) in Ndola, Zambia. PLoS One. 2020;15(4):1–19.
    OpenUrlCrossRefPubMed
  94. 94.↵
    Mburu M, Guzé MA, Ong’wen P, Okoko N, Moghadassi M, Cohen CR, et al. Evaluating the effectiveness of the HIV adolescent package of care (APOC) training on viral load suppression in Kenya. Public Health. 2019 Aug;173:146–9.
    OpenUrlPubMed
  95. 95.↵
    Chory A, Callen G, Nyandiko W, Njoroge T, Ashimosi C, Aluoch J, et al. A Pilot Study of a Mobile Intervention to Support Mental Health and Adherence Among Adolescents Living with HIV in Western Kenya. AIDS Behav. 2021;1–11.
  96. 96.↵
    Bermudez LG, Ssewamala FM, Neilands TB, Lu L, Jennings L, Nakigozi G, et al. Does Economic Strengthening Improve Viral Suppression Among Adolescents Living with HIV? Results From a Cluster Randomized Trial in Uganda. AIDS Behav. 2018 May 30;22(11):3763–72.
    OpenUrl
  97. 97.↵
    Cluver LD, Hodes RJ, Toska E, Kidia KK, Orkin FM, Sherr L, et al. ‘HIV is like a tsotsi. ARVs are your guns’: associations between HIV-disclosure and adherence to antiretroviral treatment among adolescents in South Africa. LWW; 2015.
  98. 98.↵
    Haghighat R, Toska E, Bungane N, Cluver L. The HIV care cascade for adolescents initiated on antiretroviral therapy in a health district of South Africa: a retrospective cohort study. BMC Infect Dis. 2021;21–60.
  99. 99.↵
    Ammon N, Mason S, Corkery JM. Factors impacting antiretroviral therapy adherence among human immunodeficiency virus–positive adolescents in Sub-Saharan Africa: a systematic review. Public Health Elsevier B.V.; Apr 1, 2018 p. 20–31.
  100. 100.
    Shubber Z, Mills EJ, Nachega JB, Vreeman R, Freitas M, Bock P, et al. Patient-Reported Barriers to Adherence to Antiretroviral Therapy: A Systematic Review and Meta-Analysis. Weiser SD, editor. 2016 Nov;13(11):e1002183.
    OpenUrl
  101. 101.↵
    Kim SHH, Gerver SM, Fidler S, Ward H. No Title. AIDS. 2014 Aug 24;28(13):1945–56.
    OpenUrlCrossRefPubMed
  102. 102.↵
    Kim SH, Gerver SM, Fidler S, Ward H. Adherence to antiretroviral therapy in adolescents living with HIV: systematic review and meta-analysis.
  103. 103.
    McMahon JH, Elliott JH, Bertagnolio S, Kubiak R, Jordan MR. Viral suppression after 12 months of antiretroviral therapy in low- and middle-income countries: a systematic review. Bull World Health Organ. 2013 May;91(5):377–85.
    OpenUrlCrossRefPubMedWeb of Science
  104. 104.
    Long L, Kuchukhidze S, Pascoe S, Nichols BE, Fox MP, Cele R, et al. Retention in care and viral suppression in differentiated service delivery models for HIV treatment delivery in sub-Saharan Africa: a rapid systematic review. J Int AIDS Soc. 2020 Nov;23(11):e25640.
    OpenUrlCrossRefPubMed
  105. 105.↵
    Kariminia A, Law M, Davies MA, Vinikoor M, Wools-Kaloustian K, Leroy V, et al. Mortality and losses to follow-up among adolescents living with HIV in the IeDEA global cohort collaboration. J Int AIDS Soc. 2018 Dec 1;21(12).
  106. 106.↵
    Rosen S, Fox MP, Gill CJ. Patient Retention in Antiretroviral Therapy Programs in Sub-Saharan Africa: A Systematic Review. PLoS Med. 2007 Oct;4(10):e298.
    OpenUrlCrossRefPubMed
  107. 107.
    Nyakato P, Schomaker M, Fatti G, Tanser F, Euvrard J, Sipambo N, et al. Virologic non-suppression and early loss to follow up among pregnant and non-pregnant adolescents aged 15–19 years initiating antiretroviral therapy in South Africa: a retrospective cohort study. J Int AIDS Soc. 2022 Jan 1;25(1):e25870.
    OpenUrl
  108. 108.↵
    Munyayi FK, Van Wyk B, Mayman Y. Interventions to Improve Treatment Outcomes among Adolescents on Antiretroviral Therapy with Unsuppressed Viral Loads: A Systematic Review. 2022;
  109. 109.↵
    Bam K, Rajbhandari RM, Karmacharya DB, Dixit SM. Strengthening adherence to Anti Retroviral Therapy (ART) monitoring and support: Operation research to identify barriers and facilitators in Nepal Health systems and services in low and middle income settings. BMC Health Serv Res. 2015 May 5;15(1):1–11.
    OpenUrlCrossRefPubMed
  110. 110.↵
    Brown MJ, Adeagbo O. Trauma-Informed HIV Care Interventions: Towards a Holistic Approach. Curr HIV/AIDS Rep. 2022 Mar 30;19(3):177–83.
    OpenUrl
  111. 111.↵
    Langebeek N, Gisolf EH, Reiss P, Vervoort SC, Hafsteinsdóttir TB, Richter C, et al. Predictors and correlates of adherence to combination antiretroviral therapy (ART) for chronic HIV infection: A meta-analysis. BMC Med. 2014 Aug 21;12(1):1–14.
    OpenUrlCrossRefPubMed
  112. 112.
    Kim J, Lee E, Park BJJ, Bang JH, Lee JY. Adherence to antiretroviral therapy and factors affecting low medication adherence among incident HIV-infected individuals during 2009-2016: A nationwide study. Sci Rep. 2018;8(1):1–8.
    OpenUrlCrossRefPubMed
  113. 113.
    Hudelson C, Cluver L. Factors associated with adherence to antiretroviral therapy among adolescents living with HIV/AIDS in low- and middle-income countries: a systematic review. AIDS Care. 2015 Jul 3;27(7):805–16.
    OpenUrlCrossRefPubMed
Back to top
Posted October 10, 2022.
Download PDF
Data/Code
Email
Barriers and facilitators to anti-retroviral therapy adherence among adolescents aged 10 to 19 years living with HIV in sub-Saharan Africa: A mixed-methods systematic review
Londiwe D. Hlophe, Jacques L. Tamuzi, Constance S. Shumba, Peter S. Nyasulu
medRxiv 2022.10.07.22280823; doi: https://doi.org/10.1101/2022.10.07.22280823
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area