Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Strategies and interventions to strengthen pharmacovigilance systems in low- and middle-income countries: a scoping review

View ORCID ProfileOlga Menang, Andrea de Soyres-Kuemmerle, Karen Maigetter, Christian Burri
doi: https://doi.org/10.1101/2022.12.06.22283123
Olga Menang
1Swiss Tropical and Public Health Institute, Department of Medicine, Allschwil, Switzerland
2University of Basel, Basel, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Olga Menang
  • For correspondence: olga.menang{at}swisstph.ch
Andrea de Soyres-Kuemmerle
1Swiss Tropical and Public Health Institute, Department of Medicine, Allschwil, Switzerland
2University of Basel, Basel, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Karen Maigetter
1Swiss Tropical and Public Health Institute, Department of Medicine, Allschwil, Switzerland
2University of Basel, Basel, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christian Burri
1Swiss Tropical and Public Health Institute, Department of Medicine, Allschwil, Switzerland
2University of Basel, Basel, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

ABSTRACT

Introduction In the past decades, numerous strategies have been proposed to strengthen pharmacovigilance (PV) systems in low-and-middle-income countries (LMIC). Though there has been progress, many PV systems in LMIC are still not performing adequately. The objectives of this scoping review are to describe strategies to strengthen PV in LMIC and to propose recommendations for future investments in capacity building.

Methods The review was conducted following the Joanna Briggs Institute (JBI) guidelines on conducting scoping reviews. Literature searches were performed in MEDLINE, EMBASE, Web of Science, PDQ-evidence, CINAHL and relevant websites from 1990 to January 2021. Publications included were primary studies, articles, and policy and guideline papers, describing interventions to strengthen PV in LMIC. Two reviewers independently screened titles, abstracts and full-texts, and one reviewer performed data extraction and carried out a descriptive data analysis.

Results 10,903 unique titles were screened and 152 were eligible for full text review. Of these, 57 and an additional 13 reports from grey literature fulfilled eligibility criteria for inclusion in the review. Forty-five papers (64%) described interventions aimed at increasing PV knowledge and reporting of adverse drug reactions (ADR), primarily education of healthcare professionals (HCP), alone or in combination with other interventions such as mobile and electronic reporting, enhanced passive and active surveillance. Twenty-five papers (36%) discussed interventions or strategies implemented at the national targeting different components of the national PV system, such as legal basis and guidelines for PV, PV organisation and infrastructure and PV procedures.

Conclusions Results of this review suggest that educating HCP on ADR reporting is the most common approach to build PV capacity in LMIC. Though important, education alone is insufficient and should ideally be organised within the holistic framework of strengthening national PV systems, with a focus on building capacity for advanced activities such as signal detection.

Key message There are numerous strategies and recommendations to strengthen PV systems in LMIC. However, the effective implementation of these strategies has not been extensively described in literature. This scoping review describes different interventions and strategies that have been implemented to build and improve PV capacity in systems in LMIC. An overview of current PV strengthening strategies and interventions, and when available, their outcome and impact, is useful to guide decision making for future investments in PV development in developing countries.

Trial registration Open Science Framework (https://osf.io/ge56p)

INTRODUCTION

Several low-and-middle-income countries (LMIC) are projected to receive novel vaccines and medicines in the coming years, some of which target diseases endemic in LMIC, e.g., malaria, dengue and Ebola. Therefore, pharmacovigilance (PV) systems in these countries must be efficient and functional, in order to effectively monitor the safety and effectiveness of these novel products. In the past three decades, several global and local initiatives have aimed at building PV capacity in LMIC. Earlier assessments of PV systems in LMIC showed that only few countries had performing PV systems that could detect, evaluate, and prevent medicine safety issues (1, 2). Though more recent evaluations show major improvement, (3, 4), a majority of national regulatory authorities (NRA) in LMIC have not yet attained World Health Organisation (WHO) Global Benchmarking Tool (GBT) performance maturity level 3 (stable, well-functioning and integrated regulatory system) and level 4 (regulatory systems operating at advanced level of performance and continuous improvement) (5).

Numerous initiatives have aimed to strengthen PV in LMIC in the past decades. In 2011, the WHO Global Vaccine Safety Blueprint (GVSB 1.0) proposed a three-part strategy for building vaccine PV: i) build minimum capacity for passive vaccine safety surveillance in all countries, ii) build enhanced capacity for active surveillance in countries where newly developed vaccines will be introduced, and iii) establish a global vaccine safety support structure (6). In 2013, a five-part PV enhancement strategy was proposed in a report submitted to the Safety and Surveillance Working Group of the Bill and Melinda Gates Foundation (BMGF): i) focus on a global health product pipeline, ii) implement a risk-based prioritization of candidate drugs and vaccines, iii) invert the current capacity building paradigm, iv) incorporate sustainability from outset, and v) plan scalability (2). In 2021, the WHO Global Vaccine Safety Blueprint 2.0 (GVSB2.0) proposed to move from minimal and enhanced capacity concept proposed in GVSB1.0 (6) to maturity levels, using a scale of 1 (least developed) to 4 (regulatory system operating at advanced level), to assess the level of development of monitoring and regulatory systems (7).

The slow progress of PV in LMIC not only raises questions about core challenges on the growth of PV, but also about the appropriateness of strategies and interventions used so far to develop PV. Describing these strategies and if possible, their outcomes would be useful to guide decision making for future national PV strengthening initiatives. There have been only few scoping and systematic reviews on topics related to PV globally. These include reviews on interventions to improve adverse drug reactions (ADRs) reporting among health care professionals (HCP) (8, 9) and the use of social media for PV (10, 11). However, these reviews were not solely focused on LMIC and did not describe a holistic approach to developing PV in LMIC. The objectives of this current scoping review are to describe strategies and interventions that have been implemented to strengthen national PV systems in LMIC and to recommend areas for future investments in capacity building. As PV systems in LMIC continue to mature, supported by multilateral agencies and donors, synthesised information on the different enhancement approaches will be useful for national and global PV stakeholders, including donor agencies, as they continue to build effective and sustainable PV systems.

METHODS

Research questions

  1. What interventions or strategies have been used to build and strengthen national PV systems in LMIC in the last three decades?

  2. What is the impact of these strategies and what recommendations can be made to policy-makers for implementing and/or strengthening PV systems in LMIC?

Study design

The scoping review was conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews (12) and conformed with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) (13). The protocol was developed according to the JBI scoping review template and was registered with Open Science Framework.

Eligibility criteria

Publications or papers within the last 30 years (1990 to 2021) that described strategies or intervention to establish or strengthen national PV systems in LMIC were included. Also included, were reports describing PV systems if these included interventions to develop PV. Lastly, national PV guidelines, policy and strategic documents from NRA were also included.

Information sources

This scoping review included primary research, both qualitative and quantitative, that described interventions or strategies to strengthen PV capacity, as well as policy and guideline papers from NRA. Literature searches were performed in MEDLINE, EMBASE, Web of Science, CINAHL and PDQ-Evidence. Sources of grey literature included Google Scholar, websites of technical agencies such as WHO and websites of selected NRA. A selection of NRA websites was necessary, as it was practically impossible to search all LMIC NRA websites. Therefore, websites of countries selected to participate in the next phase of the research (semi-structured interview on the same research question) were searched.

Search strategy

An initial limited search of MEDLINE, EMBASE, Epistemonikos, JBI Database of Systematic Reviews and Implementation Reports and Web of Science was undertaken to identify articles of interest. A librarian used text words contained in the titles and abstracts of relevant articles and the index terms used to describe these articles, to develop a full search strategy in MEDLINE (Appendix I). This search strategy was adapted for each database cited above using the Polyglot Search Translator (14). A simpler search strategy was used to search Google Scholar. A grey literature search was conducted according to the Canadian Agency for Drugs and Technologies in Health Guide (15) (Appendix 2).

Study selection

Identified citations from the searches were collated and uploaded into EndNote X9.3 (Clarivate Analytics, PA, USA) and duplicates removed. To facilitate study selection, a decision tree was developed and tested by two reviewers, [Olga Menang (OM) and Andrea de Soyres-Kuemmerle (ASK)], on a random sample of 100 titles and abstracts. Three batches of 100 titles and abstracts were tested until both reviewers agreed on the selection. OM reviewed 100% of titles and abstracts while ASK reviewed 15%, as selected titles and abstracts were similar for both reviewers for the first 1500 titles and abstracts screened. Two batches of 10 full-text articles were tested until agreement on article selection; both reviewers reviewed all full texts selected at title and abstract screening. The results of the search and the study inclusion process were reported in full and presented in a Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram (13).

Data extraction

Data were extracted using a data extraction tool developed for the purpose by the reviewers (Appendix III). Extraction parameters included the nature and description of the intervention, the outcome, the challenges encountered and the lessons learnt. Extracted data was reviewed by ASK and a third reviewer, Karen Maigetter (KM), and was revised as needed.

Data analysis and presentation

The extracted data were presented in a descriptive summary aligned with the objectives of the review. Data was analysed by strategy, nature and types of intervention to strengthen PV systems, and if the strategy or interventions targeted one or several components of the PV system. Risk of bias or quality appraisal across studies was not conducted, which is consistent with the Joanna Briggs Institute Methods Manual, because the scoping review method is not intended to be used to critically appraise (or appraise the risk of bias of) a cumulative body of evidence (13).

RESULTS

Study selection

The study selection flow diagram is presented in Figure 1. 10,903 unique titles were screened, of which 152 articles were selected during title and abstract screening. Of these 152 articles, 57 articles and an additional 13 articles from Google Scholar and grey literature searches fulfilled eligibility criteria for inclusion in the review (Figure 1). Appendix VI provides citation details of papers excluded on full-text examination and reasons for their exclusion.

Figure 1:
  • Download figure
  • Open in new tab
Figure 1: Study selection flow diagram

Nature of interventions and strategies

The regions and countries of papers included in the analysis of this review are presented in Appendix IV. Papers from East Asia Pacific region (9), Europe and central Asia (1), Latin America and the Caribbean (6), Middle East and North Africa (5), South Asia (17), Sub-Saharan Africa (28) and 5 papers providing global/LMIC recommendations were included. Strategies and interventions identified to strengthen PV systems in LMIC were divided into 2 categories: 1) Interventions aimed at increasing PV knowledge and reporting of adverse events (AEs) [ADRs or adverse events following immunisation (AEFI)] and, 2) Interventions aimed at building or strengthening various components of the national PV system. Details of each intervention are provided in Appendix V.

Interventions aimed at increasing PV knowledge and AE reporting

Forty-five papers described interventions aimed at increasing HCP PV knowledge and reporting of ADRs or AEFI. These included educational interventions (lectures or workshops), alone or in combination with other interventions such as mobile and electronic reporting, enhanced passive and active surveillance or providing financial incentives.

Educational interventions

Although all publications included in the analysis had an educational component, 21 out of 70 (30%) reported exclusively educational or training workshops, aimed at educating and sensitizing HCP on AE reporting and PV. Primarily provided by health institutions, these interventions were not reported as part of a broader PV strengthening scheme. The format, content, and duration varied, with duration ranging from 45 minutes (16) to several days (17). For a majority of these interventions, the content was information on ADR reporting and raising awareness on PV. In some cases, educational material such as handouts and job aids were distributed. Some interventions included practical sessions on completing AE reporting forms (18), feedback or reminders to HCP (19), and follow-up contacts to assess changes in AE reporting (20) or to address any challenges encountered (21).

There was an increase in the knowledge, attitude and practice skills of PV and ADR reporting rates for most educational interventions. After interventions, some papers reported a 12-fold increase in the number of ADRs reported and an increase in the quality of ADRs (22), or a fourteen-fold increase in ADR reporting (19), or as high as a 70-fold increase in the number of ADR reports (18). Only very few papers reported a small impact of education training. One of these was the training of naïve HIV-positive patients with limited literacy to identify and report ADRs using pictorial representation (23).

Mixed interventions

Eleven papers reported education in combination with other interventions such as financial incentives and changes in institutional ADR reporting policies. Two papers described provision of financial compensation (in addition to other changes) as an incentive for reporting ADRs in a hospital setting. For the first, ADRs increased from 29 at pre-intervention, to 277 at first intervention, to 666 at second intervention period, with an increase in the quality of reports (24). For the second, ADRs increased from 577 and to 832 after intervention, with an increase in reporting of serious ADRs and improvements in reporting compliance (25).

Other inventions included follow-up visits (26), providing feedback to HCP on ADR reporting and reviewing the quality and completeness of ADRs (27) or distributing standard operating procedures on ADR reporting, improving reporting mechanisms and supervisory visits (28). In Mozambique, in addition to training workshops on PV, a ‘yellow card’ system for spontaneous reporting of ADRs was implemented, a district focal person to facilitate communication between the health staff and the PV unit was designated and after 5 months, HCP were retrained to reinforce learnings from the first workshop (29). These papers also reported an increase in PV knowledge and number of ADRs reported.

Mobile phone and electronic reporting

Seven papers in this review described the use of mobile phones, mobile applications and electronic reporting platforms as mechanisms to increase AE reporting. For example, in Rwanda, patients were educated to report ADRs using mobile phones either by accepting a call from physicians or by calling physicians directly. Compared to the previous months where no ADRs were reported, 34 patients reported an ADR after intervention (30). Likewise, in Cameroon, 20 AEFI were received from a group reporting with a mobile phone, compared to one AEFI from the group reporting physically at the health facility (31). Other papers reported the development of mobile applications for reporting ADRs reporting (32) and supporting ADR evaluation and causality assessment(33).

At the national level, in Brazil, the national electronic reporting system, Notivisa, was replaced by VigiFlow™ and implemented at all levels of the health systems including a module for the public and another for industry. HCP were trained on PV during the process and the new tool was promoted through different platforms, resulting in a 62% increase in ADRs received by Anvisa compared to the same period for the previous year, including reports from the public (34). Likewise, Mexico implemented VigiFlow™, and e-reporting throughout the country, enabling ADR reporting from health services and industry, as well as analysis of safety data (35).

Passive and active surveillance

Six papers in the current review reported implementation of enhanced passive and/or active surveillance as mechanisms to increase reporting and collect more reliable PV data. These papers reported an increase in the number, timeliness and quality of ADRs collected (36-38) even though one paper reported that the numbers reduced in subsequent months (39).

Interventions aimed at strengthening various components of the national PV system

Twenty-five papers discussed interventions or strategies implemented at the national level that aimed to strengthen different components of the PV system. For example, PV in India was implemented in a stepwise approach, targeting different components and stakeholders of the PV systems. These steps included: i) development and revision of PV guidelines; ii) establishing and training national safety and AEFI committees; iii) setting up regional ADR monitoring centres; vi) developing communication tools and guidelines; v) integrating PV in public health programs (PHP); vi) android mobile application and toll-free helpline to enable reporting of ADRs; and vii) safety communication to stakeholders and the public though different media (40-42). A similar comprehensive approach was reported in Vietnam, and within 7 years of the creation of the national PV centre, both the quality and the quantity of ADRs improved (43).

In Malawi, the national PV system was enhanced through a combination of strategies targeting the PV organisation and infrastructure, PV procedures and AE reporting, national coordination of PV activities and training of PV personnel and HCP. During the 18 months of implementation, 443 HCP at 61 healthcare facilities were trained in PV and the ADRs and AEFI increased from 22 at pre-intervention to 228 (44). Other interventions identified at the national level included creation of the national PV centre, including PV in curriculum of medicine, pharmacy, and dentistry schools, and building collaborations between NRA and PHP (45), or development of PV policies and tools; establishment and training national safety review committees; and establishment of country coordination teams (46).

In 2019, the National Drug Authority of Uganda published a 5-year (2018 to 2022) strategic plan to develop PV. The plan defined four strategic areas of development: i) build PV infrastructure and technical capacity; ii) PV policy enhancement; iii) collaboration and information exchange; and iv) visibility and awareness of PV and safety related information (47). In 2020, the Nigerian National Pharmacovigilance Policy was revised, providing a framework for the implementation of the policy, covering the entire scope of PV, including stakeholders, institutionalisation of PV policy documents, PV structures, human resources, industry, funding and monitoring and evaluation (48). In Namibia, results of a qualitative survey recommended several strategies on the integration of PV systems in public health: i) enhancing reporting by integrating PV into the daily practice of HCP; ii) establishing policies addressing PV activities; iii) capacity-building through continuing professional education and support, advocacy, stakeholder engagement; iv) incentivising and recognition for PV performance; and v) facility-based policies for universal and inclusive reporting (49).

Also identified in this review were global recommendations on strengthening national PV systems. These included WHO’s Project 3-S (Smart Safety Surveillance), aimed at optimizing post marketing surveillance of priority medicines and vaccines in LMIC. Six core strategies were proposed: i) adopt a stepwise approach with an initial pilot for three new products; ii) leverage available resources from WHO partners and national PV centres; iii) partnership with industry; iv) develop a holistic country plan for PV as part of medicines regulation; v) collaboration with other global initiatives; and vi) build PV infrastructure progressively, from minimum to mid-range and advanced capacity (50). In 2018, Elshafie et al., proposed different strategies to encourage the introduction and sustain the advancement of robust PV systems in developing countries: i) enhancing spontaneous reporting though a combination of interventions; ii) training HCP and incorporation PV into educational institutions and curricula of HCP; iii) actively engaging the public to participate in PV; iv) facilitating ADR reporting by simplifying the process and enabling mobile and electronic reporting and enabling adequate data management and communication of safety signals; v) establish well-organized healthcare systems, with effective policies and strict drug regulations; and vi) implement regulations and guidelines for industry (51). Lastly, in 2021, PhArmacoVigilance Africa (PAVIA) released the “Guide for Effective Implementation of the Pharmacovigilance Policy in Resource Limited Settings”, describing critical steps and actions for effective implementation of a national PV policy: stakeholders’ engagement, statutory endorsement, institutionalization of the PV policy, addressing the roles and responsibilities of stakeholders and securing funding (52).

DISCUSSION

This scoping review was the first, to the best of our knowledge, to describe the strategies and interventions to improve PV capacity in LMIC in the last three decades. Two-thirds of papers identified in this review described interventions aimed at improving ADR reporting. Educating HCP on PV and ADR reporting is an effective method to change their attitude towards reporting, and provides them with an understanding of the ADR reporting process and other issues associated with underreporting of ADRs, such as fear of litigation (53, 54). It is not surprising, therefore, given the major role played by HCP in reporting of ADRs, that a majority of interventions identified in this review had an educational component. Findings revealed that these PV educational or training workshops were very effective in improving PV knowledge with a corresponding increase in the number and quality of ADR reports. This is in line with numerous previous scoping or systematic reviews aimed at mapping interventions to improve ADR reporting among HCP (8, 9).

Several papers also described training of other PV stakeholders such as the national safety review committees (46, 55-57) and consumers (30, 37, 58, 59). National safety review committees perform causality assessments of serious cases and support analysis of national PV data. Nevertheless, the timing of the causality assessment is important and their work is dependent on stable funding. ADRs from patients and consumers are an important source of information on the safety and effectiveness of medicines and provides useful information for improving drug safety and health care from a different perspective (60). Consequently, there is a global trend toward educating and empowering patients and consumers to report ADRs, and studies have established the significant contribution of consumer reporting to signal detection (59, 61, 62).

Electronic reporting of ADRs is one of the main strategies used globally to improve spontaneous reporting of ADRs by HCP (63). In many developed countries, patients and consumers can directly report ADRs through web-based platforms. As seen in this review, NRA in LMIC are also moving towards implementing web-based electronic reporting and data management platforms such as Vigibase™ (34, 35) to handle data at the national level. Likewise, mobile phones and mobile phone applications are increasingly used to expand and facilitate ADR reporting in HCP as demonstrated by several papers in this review (32, 33, 36, 39, 64). Enhance passive and active surveillance are PV methods usually implemented during new medicinal production to increase and improve the collection and quality of PV data. The introduction of novel vaccines and drugs in LMIC represents an opportunity to strengthen national safety surveillance systems. Significant technical and financial efforts from national and global PV stakeholders are directed towards enhancing national PV systems prior to medicine introduction. In addition to other interventions, training of HCP and sometimes, patients, on PV and ADRs is one of the key preparatory activities of vaccine introduction preparations, often implemented through passive, enhanced passive and active surveillance as identified in this review (36, 37, 39).

Educating HCP and other PV actors on different aspects of PV and ADR reporting remains an essential aspect of building national PV capacity and numerous organisations and NRA provide diverse PV training programmes. These types of interventions are often simple to implement and can attain HCP at all levels of the health system. However, mechanisms for measuring their long-term impact are needed and there is a need for retraining and continuous education to sustain acquired knowledge. Several papers in this review showed that the increase in ADRs observed after the intervention periods was not sustained in subsequent months (18, 22, 39), and only few publications mentioned retraining of HCP (29), mentorship and supervisory visits (28, 45, 64) as part of the interventions. Therefore, education alone does not suffice and could have more impact in combinations with other techniques and implemented within the broader context of strengthening national PV systems.

About one-third of papers identified presented a more comprehensive structured approach to develop national PV systems. When interventions are implemented in a structured manner and are supported by validated tools such as the WHO GBT, different components of the PV system can be systematically strengthened, by formulating institutional development plans that address identified gaps, prioritize interventions and facilitate monitoring of progress and achievements (5). As shown in the Results, the stepwise implementation of PV in India, targeting the key indicators of the PV system (40-42), resulted in the creation of a performant PV system assessed at maturity level 4 for several functions such as clinical trial oversight, marketing authorization and vigilance. India contributes 3% ADRs to the global safety database, with a completeness score of 0.93 out of 1. Likewise, in Malawi, a step-wise approach targeting different PV components, with the collaboration of different technical agencies, enabled the creation of a national PV system, with guidelines, organisational structure, PV infrastructure and mechanisms for ADR reporting and other PV processes such as active surveillance (44, 56, 65).

Though several papers described different strengthening interventions at the national level with positive results, it was more difficult to identify national strategies with holistic operational plans in line with recent WHO recommendations, such as identified in Uganda (47), Tanzania (66) and Nigeria (48, 67). A recent publication has described the key strategies, outcomes, challenges and lessons learnt on building Eritrean national PV system. The key strategies for success included: i) establishing organisational capacity and capability for PV; ii) integrating PV into all PHP; iii) integrating PV into academia; iv) maximizing ADR reporting through massive training, recognition and feedback, appointing PV focal points, a combination of passive, stimulated passive and active surveillance, amongst others; and v) signal detection and assessment. As a result, Eritrea has been rated among the top reporting countries in Africa and the centre has detected 30 safety signals, and achieved maturity level 3 on the WHO GBT (68). Such strategic documents on strengthening PV systems provide valuable information to national and global PV initiatives and stakeholders on developing PV systems in LMIC. These strategies can be adapted to specific local contexts with feasible operational plans, providing a framework to build, track and continuously improve all components of the national PV system.

Overall, findings from this review revealed similar strategies and interventions aimed at building functional PV systems across several LMIC. The main strategies identified were: i) educating HCP and increasing patients, on PV and reporting of ADRs; ii) facilitating reporting of ADRs using different mechanisms such as mobile and electronic reporting, enhanced passive and active surveillance; iii) integrating PV in PHP and developing mechanisms for coordination and information exchange; iv) establishing and training national safety committees and developing capacity for data analysis; v) developing PV policies, guidelines and tools; vi) including PV in curriculum of HCP. There was a paucity of articles describing capacity building for more advanced PV activities such as establishing quality systems with procedures for aggregate data analysis and signal detection. Nevertheless, interventions identified, whether alone or in combination, resulted in either an increase in PV knowledge, ADR reporting or improvement of national PV systems or capacity, though not necessarily resulting in functional PV systems.

Challenges in strengthening ADR reporting and PV systems

The main challenges identified in papers included in this review were:

  1. Underreporting of ADRs, associated with numerous factors including lack of/or inadequate knowledge of reporting procedures, unavailability of reporting forms, lack of motivation, and lack of adequate methods of communication (29, 40, 58, 59, 69).

  2. Lack of financial and human resources dedicated to PV, primarily because PV activities in numerous LMIC rely largely on external funds. When these funds run out, PV activities also stagnate (45, 46).

  3. Insufficient national coordination of PV activities and inadequate collaboration and coordination between major national PV stakeholders hinders PV development (43, 44, 46).

  4. Interventions were not sustained because often, there were no long-term plans and continuous funding. Only few publications mentioned retraining of HCP, mentorship and supervisory visits and these often come at a cost (22, 26, 29, 70).

Lessons learnt and implications of the findings for research

As revealed in the current and in previous reviews, a majority of interventions to improve spontaneous reporting in LMIC were based on training workshops, often single-centre short-term activities. The importance of spontaneous reporting and educational interventions to improve passive surveillance cannot be over-emphasized. However, these should ideally be continuous, and combined with other activities aimed at developing national PV systems. Findings show a gap in more advanced PV activities such as aggregate data analysis, signal detection, active safety surveillance and risk management and communication. With novel medicines increasingly introduced in low-resource settings, effective and adequate safety surveillance will depend on these countries’ capacity for signal detection, analysis and interpretation of safety data. Therefore, building capacity for advanced PV activities is an important area of investment for future PV strengthening initiatives. Lastly, improving PV systems should ideally be holistic, targeting all components of the PV system, with plans for sustainability, and procedures to periodically assess improvements with validated tools.

Limitations of the research

This review aimed to identify strategies and interventions to build and strengthen PV systems in LMIC. The search strategy mainly identified interventions aimed at improving ADR knowledge and reporting amongst HCP, with fewer articles identified on strategic approaches to strengthen national PV systems in LMIC. This is probably because results of such strategies and activities are not often published in scientific literature and are uncommonly publicly available. In addition, there is also a possible publication bias with the effect that only reasonably successful interventions at the national level would likely be submitted for publication. A limited search of selected national regulatory websites was conducted, because an extended search of NRA websites would have required significant resources, beyond the scope of this research, as discussed in Methods.

CONCLUSIONS

Findings from this review showed that a combination of interventions such as education, financial incentives, mobile and electronic reporting, and enhanced passive and active surveillance significantly improved ADR reporting. Educating HCP on PV was the most common approach to build PV capacity in LMIC. However, education alone is insufficient in that it must be continuous, and ideally organised within the framework of strengthening the national PV system. A structured approach, targeting all PV components, with a particular focus on building capacity for more complex PV activities, is judicious and provides a framework to continuously build, track and improve national PV strengthening activities.

Data Availability

All data produced in the present work are contained in the manuscript and in the additional file

STATEMENTS AND DECLARATIONS

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Conflict of interest

There are no conflicts of interest this research.

Ethics approval

The Ethikkommission Nordwest-und Zentralschweiz (EKNZ) has confirmed that no ethical approval was required.

Author’s contributions

Olga Menang: Conceptualization, data curation, formal analysis, methodology, project administration, resources, visualization, writing – original draft, writing – review & editing

Christian Burri, Andrea Kuemmerle and Karen Maigetter contributed to the study design and the analysis and interpretation of the data. All authors read and approved the final manuscript.

ACKNOWLEDGEMENTS

The authors also wish to thank Andy Stergachis (Director, Global Medicines Program & Biomedical Regulatory Affairs Program, University of Washington School of Pharmacy) and Günther Fink (Head, Household Economics and Health System Research Unit, SwissTPH) for reviewing this manuscript. Thanks to Hannah Ewald (University Medical Library, University of Basel, Switzerland) for advice on the research protocol, developing the search strategy and for conducting the literature searches. Thanks to Niranjan Bhat (PATH) and Emmanuel Mpinga Kabengele (Institute of Global Health at the University of Geneva) for reviewing the scoping review.

REFERENCES

  1. 1.↵
    SPS. Safety of Medicine in Sub-Saharan Africa: Assessment of Pharmacovigilance Systems and their Performance. Submitted to the US Agency for International Development by the Strengthening Pharmaceutical Systems (SPS) Program. Arlington, VA: Management Sciences for Health. 2011.
  2. 2.↵
    Bollyky T, Stergachis A. A Strategy for Improving Post-Market Safety Surveillance of Drugs & Vaccines in Low- and Middle-Income Countries. A report of the Safety Surveillance Working Group. Bill & Melinda Gates Foundation.; 2013.
  3. 3.↵
    Barry A, Olsson S, Minzi O, Bienvenu E, Makonnen E, Kamuhabwa A, et al. Comparative Assessment of the National Pharmacovigilance Systems in East Africa: Ethiopia, Kenya, Rwanda and Tanzania. Drug Safety. 2020:1–12.
  4. 4.↵
    Garashi HY, Steinke DT, Schafheutle EI. A systematic review of pharmacovigilance systems in developing countries using the WHO pharmacovigilance indicators. Therapeutic Innovation & Regulatory Science. 2022:1–27.
  5. 5.↵
    Khadem Broojerdi A, Baran Sillo H, Ostad Ali Dehaghi R, Ward M, Refaat M, Parry J. The world Health organization global benchmarking tool an instrument to strengthen medical products regulation and promote universal health coverage. Frontiers in medicine. 2020:457.
  6. 6.↵
    WHO. Global Vaccine Safety Blueprint 2012.
  7. 7.↵
    WHO. Global vaccine safety blueprint 2.0 (GVSB2.0) 2021-2023. Geneva: World Health Organization; 2021. Licence: CC BY-NC-SA 3.0 IGO. 2021.
  8. 8.↵
    Khalili M, Mesgarpour B, Sharifi H, Daneshvar Dehnavi S, Haghdoost A. Interventions to improve adverse drug reaction reporting: A scoping review. Pharmacoepidemiology & Drug Safety. 2020.
  9. 9.↵
    Paudyal V, Al-Hamid A, Bowen M, Hadi MA, Hasan SS, Jalal Z, et al. Interventions to improve spontaneous adverse drug reaction reporting by healthcare professionals and patients: systematic review and meta-analysis. Expert Opinion on Drug Safety. 2020;19(9):1173–91.
    OpenUrl
  10. 10.↵
    Lee J-Y, Lee Y-S, Kim DH, Lee HS, Yang BR, Kim MG. The use of social media in detecting drug safety–related new black box warnings, labeling changes, or withdrawals: scoping review. JMIR public health and surveillance. 2021;7(6):e30137.
    OpenUrl
  11. 11.↵
    Tricco AC, Zarin W, Lillie E, Jeblee S, Warren R, Khan PA, et al. Utility of social media and crowd-intelligence data for pharmacovigilance: a scoping review. BMC Med Inform Decis Mak. 2018;18(1):1–14.
    OpenUrlPubMed
  12. 12.↵
    1. Aromataris E,
    2. Munn Z (Editors)
    Peters M, Godfrey C, McInerney P, Munn Z, Tricco A, Khalil H. Chapter 11: Scoping Reviews (2020 version). In: Aromataris E, Munn Z (Editors). Joanna Briggs Institute Reviewer’s Manual, JBI, 2020. 2020 [
  13. 13.↵
    Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Annals of internal medicine. 2018;169(7):467–73.
    OpenUrlCrossRefPubMed
  14. 14.↵
    Clark JM, Sanders S, Carter M, Honeyman D, Cleo G, Auld Y, et al. Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial. Journal of the Medical Library Association: JMLA. 2020;108(2):195.
    OpenUrl
  15. 15.↵
    CADTH. Grey matters: a practical tool for searching health-related grey literature 2015 [
  16. 16.↵
    Sanghavi DR, Dhande PP, Pandit VA. Perception of pharmacovigilance among doctors in a tertiary care hospital: influence of an interventional lecture. International Journal of Risk & Safety in Medicine. 2013;25(4):197–204.
    OpenUrl
  17. 17.↵
    MacDonald NE, Guichard S, Amarasinghe A, Balakrishnan MR, Inter-country SWP. Strengthening of causality assessment of adverse events following immunization in the WHO South East Asia and Western Pacific regions: Lessons from the 2014 SEAR inter-country workshop. Vaccine. 2015;33(48):6902–7.
    OpenUrl
  18. 18.↵
    Varallo FR, Planeta CS, Mastroianni PC. Effectiveness of pharmacovigilance: multifaceted educational intervention related to the knowledge, skills and attitudes of multidisciplinary hospital staff. Clinics (Sao Paulo, Brazil). 2017;72(1):51–7.
    OpenUrl
  19. 19.↵
    Morales Rios O, Jasso Gutierrez L, Talavera JO, Tellez-Rojo MM, Olivar Lopez V, Garduno Espinosa J, et al. A comprehensive intervention for adverse drug reactions identification and reporting in a Pediatric Emergency Department. International Journal of Clinical Pharmacy. 2016;38(1):80–7.
    OpenUrl
  20. 20.↵
    Deepalakshmi M, Kumar P, Arun KP, Ponnusankar S. Impact of continuing pharmacy education on the knowledge, attitude and practice of community pharmacists about ADR monitoring and reporting. Indian Journal of Pharmaceutical Sciences. 2019;81(4):633–9.
    OpenUrl
  21. 21.↵
    Alraie NA, Saad AA, Sabry NA, Farid SF. Adverse drug reactions reporting: a questionnaire-based study on Egyptian pharmacists’ attitudes following an awareness workshop. Journal of Evaluation in Clinical Practice. 2016;22(3):349–55.
    OpenUrl
  22. 22.↵
    Vo TH, Dang TN, Nguyen TT, Nguyen DT. An Educational Intervention to Improve Adverse Drug Reaction Reporting: An Observational Study in a Tertiary Hospital in Vietnam. Archives of Pharmacy Practice. 2020;11(3):32–7.
    OpenUrl
  23. 23.↵
    Gebreyohannes EA, Bhagavathula AS, Abegaz TM, Abebe TB, Belachew SA, Tegegn HG, et al. The effectiveness of pictogram intervention in the identification and reporting of adverse drug reactions in naive HIV patients in Ethiopia: a cross-sectional study. HIV/AIDS Research and Palliative Care. 2019;11:9–16.
    OpenUrl
  24. 24.↵
    Chang F, Xi Y, Zhao J, Zhang X, Lu Y. A time series analysis of the effects of financial incentives and mandatory clinical applications as interventions to improve spontaneous adverse drug reaction reporting by hospital medical staff in China. Journal of Evaluation in Clinical Practice. 2017;23(6):1316–21.
    OpenUrl
  25. 25.↵
    Fang H, Lin X, Zhang J, Hong Z, Sugiyama K, Nozaki T, et al. Multifaceted interventions for improving spontaneous reporting of adverse drug reactions in a general hospital in China. BMC Pharmacology & Toxicology. 2017;18(1):49.
    OpenUrl
  26. 26.↵
    Cortes-Serra N, Saravia R, Grageda RM, Apaza A, Gonzalez JA, Rios B, et al. Strengthening the Bolivian pharmacovigilance system: New surveillance strategies to improve care for Chagas disease and tuberculosis. PLoS Neglected Tropical Diseases [electronic resource]. 2020;14(9):e0008370.
    OpenUrl
  27. 27.↵
    Nkonde K J.C. M, M.D. M, R.S. S. Implementation of interventions to strengthen the pharmacovigilance surveillance system at Dr George Mukhari Academic Hospital, Gauteng Province (Doctoral dissertation, Sefako Makgatho Health Sciences University) 2017.
  28. 28.↵
    Terblanche A, Meyer JC, Godman B, Summers RS. Impact of a pharmacist-driven pharmacovigilance system in a secondary hospital in the Gauteng Province of South Africa. Hospital practice (1995) Hospital practice. 2018;46(4):221–8.
    OpenUrl
  29. 29.↵
    Sevene E, Mariano A, Mehta U, Machai M, Dodoo A, Vilardell D, et al. Spontaneous adverse drug reaction reporting in rural districts of Mozambique. Drug Safety. 2008;31(10):867–76.
    OpenUrlPubMed
  30. 30.↵
    Kadima NJ, Nyiranteziryayo R, Umumararungu T, Adedeji AA. Use of mobile phones for patient self-reporting adverse drug reactions: A pilot study at a tertiary hospital in Rwanda. Health and Technology. 2020.
  31. 31.↵
    Tsafack M, Ateudjieu J. Improving community based AEFI (Adverse Events Following Immunization) reporting rate through telephone “beep” in a Cameroon health district: a randomized field trial. The Pan African medical journal. 2015;22:351.
    OpenUrl
  32. 32.↵
    Prakash J, Joshi K, Malik D, Mishra O, Sachan A, Kumar B, et al. “ADR PvPI” Android mobile app: Report adverse drug reaction at any time anywhere in India. Indian Journal of Pharmacology. 2019;51(4):236–42.
    OpenUrl
  33. 33.↵
    Ithnin M, Mohd Rani MD, Abd Latif Z, Kani P, Syaiful A, Nor Aripin KN, et al. Mobile App Design, Development, and Publication for Adverse Drug Reaction Assessments of Causality, Severity, and Preventability. JMIR MHealth and UHealth. 2017;5(5):e78.
    OpenUrl
  34. 34.↵
    Vogler M, Ricci Conesa H, de Araujo Ferreira K, Moreira Cruz F, Simioni Gasparotto F, Fleck K, et al. Electronic Reporting Systems in Pharmacovigilance: The Implementation of VigiFlow in Brazil. Pharmaceutical Medicine. 2020;34(5):327–34.
    OpenUrl
  35. 35.↵
    Reports U. Cofepris boosts ability to receive and analyse adverse drug reaction reports. Uppsala Reports. 2020(83):22–3.
  36. 36.↵
    Kabanywanyi AM, Mulure N, Migoha C, Malila A, Lengeler C, Schlienger R, et al. Experience of safety monitoring in the context of a prospective observational study of artemether-lumefantrine in rural Tanzania: lessons learned for pharmacovigilance reporting. Malaria Journal. 2010;9:205.
    OpenUrl
  37. 37.↵
    Rouamba T, Sondo P, Derra K, Nakanabo-Diallo S, Bihoun B, Rouamba E, et al. Optimal Approach and Strategies to Strengthen Pharmacovigilance in Sub-Saharan Africa: A Cohort Study of Patients Treated with First-Line Artemisinin-Based Combination Therapies in the Nanoro Health and Demographic Surveillance System, Burkina Faso. Drug Design, Development and Therapy. 2020;14:1507.
    OpenUrl
  38. 38.↵
    Bravo-Alcantara P, Perez-Vilar S, Molina-Leon HF, Sturkenboom M, Black S, Zuber PLF, et al. Building capacity for active surveillance of vaccine adverse events in the Americas: A hospital-based multi-country network. Vaccine. 2018;36(3):363–70.
    OpenUrl
  39. 39.↵
    Ndiaye JA, Diallo I, Y Nd, Kouevidjin E, Aw I, Tairou F, et al. Evaluation of Two Strategies for Community-Based Safety Monitoring during Seasonal Malaria Chemoprevention Campaigns in Senegal, Compared with the National Spontaneous Reporting System. Pharmaceutical Medicine. 2018;32(3):189–200.
    OpenUrl
  40. 40.↵
    Joshi J, Das MK, Polpakara D, Aneja S, Agarwal M, Arora NK. Vaccine Safety and Surveillance for Adverse Events Following Immunization (AEFI) in India. Indian Journal of Pediatrics. 2018;85(2):139–48.
    OpenUrlCrossRefPubMed
  41. 41.
    Meher BR. Vaccine pharmacovigilance in India: Current context and future perspective. Indian Journal of Pharmacology. 2019;51(4):243–7.
    OpenUrl
  42. 42.↵
    Kalaiselvan V, Thota P, Singh GN. Pharmacovigilance Programme of India: Recent developments and future perspectives. Indian Journal of Pharmacology. 2016;48(6):624–8.
    OpenUrl
  43. 43.↵
    Nguyen KD, Nguyen PT, Nguyen HA, Roussin A, Montastruc JL, Bagheri H, et al. Overview of Pharmacovigilance System in Vietnam: Lessons Learned in a Resource-Restricted Country. Drug Safety. 2018;41(2):151–9.
    OpenUrl
  44. 44.↵
    Jusot V, Chimimba F, Dzabala N, Menang O, Cole J, Gardiner G, et al. Enhancing Pharmacovigilance in Sub-Saharan Africa Through Training and Mentoring: A GSK Pilot Initiative in Malawi. Drug Safety. 2020;43(6):583–93.
    OpenUrl
  45. 45.↵
    Nzolo D, Kuemmerle A, Lula Y, Ntamabyaliro N, Engo A, Mvete B, et al. Development of a pharmacovigilance system in a resource-limited country: the experience of the Democratic Republic of Congo. Therapeutic Advances in Drug Safety. 2019;10:2042098619864853.
  46. 46.↵
    Diomandé FVK, Yameogo TM, Vannice KS, Preziosi MP, Viviani S, Ouandaogo CR, et al. Lessons Learned From Enhancing Vaccine Pharmacovigilance Activities During PsA-TT Introduction in African Countries, 2010-2013. Clinical Infectious Diseases. 2015;61:S459–S66.
    OpenUrlCrossRefPubMed
  47. 47.↵
    NDA. Monitoring safety of medicines in Uganda: strategies for better pharmacovigilance systems in settings with limited resources 2020.
  48. 48.↵
    Health Mo. Nigerian national pharmacovigilance policy and implementation framework. 2020.
  49. 49.↵
    Adenuga BA, Kibuule D, Bamitale KDS, Rennie TW. Effective integration of pharmacovigilance systems at public health facilities in resource-limited settings: A qualitative study. Research In Social & Administrative Pharmacy. 2020;16(8):1111–6.
    OpenUrl
  50. 50.↵
    WHO. Safety of medicines: priming resource-limited countries for pharmacovigilance. WHO Drug Information. 2017;31(4):575–80.
    OpenUrl
  51. 51.↵
    Elshafie S, Roberti AM, Zaghloul I. Pharmacovigilance in developing countries (part II): a path forward. International Journal of Clinical Pharmacy. 2018;40(4):764–8.
    OpenUrl
  52. 52.↵
    PAVIA. PAVIA Guide for Effective Implementation of the Pharmacovigilance Policy in Resource Limited Settings (2021) PAVIA-EDCTP sponsored Project 2021.
  53. 53.↵
    Perez B, Knych SA, Weaver SJ, Liberman A, Abel EM, Oetjen D, et al. Understanding the barriers to physician error reporting and disclosure: a systemic approach to a systemic problem. Journal of patient safety. 2014;10(1):45–51.
    OpenUrl
  54. 54.↵
    Nadew SS, Beyene KG, Beza SW. Adverse drug reaction reporting practice and associated factors among medical doctors in government hospitals in Addis Ababa, Ethiopia. PLoS ONE [Electronic Resource]. 2020;15(1):e0227712.
    OpenUrl
  55. 55.↵
    MacDonald NE, Guichard S, Arora N, Menning L, Wilhelm E, Inter-country Sear Workshop Participants CE. Lessons on causality assessment and communications from the 2019 South-East Asia Regional (SEAR) workshop on inter-country expert review of selected Adverse Events Following Immunization (AEFI) cases. Vaccine. 2020;38(32):4924–32.
    OpenUrl
  56. 56.↵
    WHO. Vaccine pharmacovigilance readiness for malaria vaccine implementation. 2018;93(3):17–32.
    OpenUrl
  57. 57.↵
    Hartigan-Go K, Roces MC, Habacon CA, Mansoor O, Shin S. Developing an immunization safety surveillance system in the Philippines. Bulletin of the World Health Organization. 2000;78(9):1166.
    OpenUrlPubMed
  58. 58.↵
    Poirot E, Soble A, Ntshalintshali N, Mwandemele A, Mkhonta N, Malambe C, et al. Development of a pharmacovigilance safety monitoring tool for the rollout of single low-dose primaquine and artemether-lumefantrine to treat Plasmodium falciparum infections in Swaziland: a pilot study. Malaria Journal. 2016;15(1):384.
    OpenUrl
  59. 59.↵
    Hartigan-Go K. Empowering consumers as contributors for health product safety: lessons from the Philippines. Drug Safety. 2015;38(4):329–35.
    OpenUrl
  60. 60.↵
    Weigmann K. Consumer reporting of adverse drug reactions: Systems that allow patients to report side effects of the drugs they are taking have yielded valuable information for improving drugs safety and health care. EMBO reports. 2016;17(7):949–52.
    OpenUrlFREE Full Text
  61. 61.↵
    Härmark L, Van Hunsel F, Grundmark B. ADR reporting by the general public: lessons learnt from the Dutch and Swedish systems. Drug safety. 2015;38(4):337–47.
    OpenUrlCrossRefPubMed
  62. 62.↵
    Inácio P, Cavaco A, Airaksinen M. The value of patient reporting to the pharmacovigilance system: a systematic review. British journal of clinical pharmacology. 2017;83(2):227–46.
    OpenUrlCrossRefPubMed
  63. 63.↵
    Fukushima AR, Iessa N, Balakrishnan MR, Pal SN. Smartphone-based mobile applications for adverse drug reactions reporting: global status and country experience. BMC Med Inform Decis Mak. 2022;22(1):1–20.
    OpenUrl
  64. 64.↵
    Ateudjieu J, Stoll B, Nguefack-Tsague G, Tchangou C, Genton B. Vaccines safety; effect of supervision or SMS on reporting rates of adverse events following immunization (AEFI) with meningitis vaccine (MenAfriVac TM): a randomized controlled trial. Vaccine. 2014;32(43):5662–8.
    OpenUrl
  65. 65.↵
    Stegmann J-U, Jusot V, Menang O, Gardiner G, Vesce S, Volpe S, et al. Challenges and lessons learned from four years of planning and implementing pharmacovigilance enhancement in sub-Saharan Africa. BMC Public Health. 2022;22(1):1–12.
    OpenUrlCrossRef
  66. 66.↵
    TMDA T. Tanzania Medicines and Medical Devices Authority - The National Pharmacovigilance Roadmap 2019 - 2023 2021.
  67. 67.↵
    Nigeria N. The National Agency for Food and Drug Administration and Control (NAFDAC) Strategic Plan 2018 - 2023. In: NAFDAC, editor. 2019.
  68. 68.↵
    Russom M, Bahta I, Debesai M. Eritrean Pharmacovigilance System: Key Strategies, Success Stories, Challenges and Lessons Learned. Drug safety. 2021;44(10):1021–32.
    OpenUrl
  69. 69.↵
    Khalili H, Mohebbi N, Hendoiee N, Keshtkar AA, Dashti-Khavidaki S. Improvement of knowledge, attitude and perception of healthcare workers about ADR, a pre- and post-clinical pharmacists’ interventional study. BMJ Open. 2012;2:e000367.
    OpenUrlAbstract/FREE Full Text
  70. 70.↵
    Mehta U, Durrheim D, Mabuza A, Blumberg L, Allen E, Barnes KI. Malaria pharmacovigilance in Africa: lessons from a pilot project in Mpumalanga Province, South Africa. Drug Safety. 2007;30(10):899–910.
    OpenUrlCrossRefPubMedWeb of Science
Back to top
PreviousNext
Posted December 06, 2022.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Strategies and interventions to strengthen pharmacovigilance systems in low- and middle-income countries: a scoping review
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Strategies and interventions to strengthen pharmacovigilance systems in low- and middle-income countries: a scoping review
Olga Menang, Andrea de Soyres-Kuemmerle, Karen Maigetter, Christian Burri
medRxiv 2022.12.06.22283123; doi: https://doi.org/10.1101/2022.12.06.22283123
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Strategies and interventions to strengthen pharmacovigilance systems in low- and middle-income countries: a scoping review
Olga Menang, Andrea de Soyres-Kuemmerle, Karen Maigetter, Christian Burri
medRxiv 2022.12.06.22283123; doi: https://doi.org/10.1101/2022.12.06.22283123

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Public and Global Health
Subject Areas
All Articles
  • Addiction Medicine (349)
  • Allergy and Immunology (668)
  • Allergy and Immunology (668)
  • Anesthesia (181)
  • Cardiovascular Medicine (2648)
  • Dentistry and Oral Medicine (316)
  • Dermatology (223)
  • Emergency Medicine (399)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (942)
  • Epidemiology (12228)
  • Forensic Medicine (10)
  • Gastroenterology (759)
  • Genetic and Genomic Medicine (4103)
  • Geriatric Medicine (387)
  • Health Economics (680)
  • Health Informatics (2657)
  • Health Policy (1005)
  • Health Systems and Quality Improvement (985)
  • Hematology (363)
  • HIV/AIDS (851)
  • Infectious Diseases (except HIV/AIDS) (13695)
  • Intensive Care and Critical Care Medicine (797)
  • Medical Education (399)
  • Medical Ethics (109)
  • Nephrology (436)
  • Neurology (3882)
  • Nursing (209)
  • Nutrition (577)
  • Obstetrics and Gynecology (739)
  • Occupational and Environmental Health (695)
  • Oncology (2030)
  • Ophthalmology (585)
  • Orthopedics (240)
  • Otolaryngology (306)
  • Pain Medicine (250)
  • Palliative Medicine (75)
  • Pathology (473)
  • Pediatrics (1115)
  • Pharmacology and Therapeutics (466)
  • Primary Care Research (452)
  • Psychiatry and Clinical Psychology (3432)
  • Public and Global Health (6527)
  • Radiology and Imaging (1403)
  • Rehabilitation Medicine and Physical Therapy (814)
  • Respiratory Medicine (871)
  • Rheumatology (409)
  • Sexual and Reproductive Health (410)
  • Sports Medicine (342)
  • Surgery (448)
  • Toxicology (53)
  • Transplantation (185)
  • Urology (165)