ABSTRACT
Objective This study aims to evaluate the impact of a primary care nurse-practitioner-led clinic model piloted in British Columbia (Canada) on patients’ health and care experience.
Design The study relies on a quasi-experimental longitudinal design based on a pre-and-post survey of patients receiving care in NP-Led clinics. The pre-rostering survey (T0) was focused on patients’ health status and care experiences preceding being rostered to the NP clinic. One year later, patients were asked to complete a similar survey (T1) focused on the care experiences with the NP clinic.
Setting To solve recurring problems related to poor primary care accessibility, British Columbia opened four pilot NP-led clinics in 2020. Each clinic has the equivalent of approximately six full-time NPs, four other clinicians plus support staff. Clinics are located in four cities ranging from core urban to peri rural.
Participants Recruitment was conducted by the clinic’s clerical staff or by their care provider. A total of 437 usable T0 surveys and 254 matched and usable T1 surveys were collected.
Primary outcome measures The survey instrument was focused on five core dimensions of patients’ primary care experience (accessibility, continuity, comprehensiveness, responsiveness, and outcomes of care) as well as on the SF-12 Short-form Health Survey.
Results Scores for all dimensions of patients’ primary care experience increased significantly: Accessibility (T0=5.9, T1=7.9, p<0.000), Continuity (T0=5.5, T1=8.8, p<0.000), Comprehensiveness (T0=5.6, T1=8.4, p<0.000), Responsiveness (T0=7.2, T1=9.5, p<0.000), Outcomes of care (T0=5.0, T1=8.3, p<0.000). SF-12 Physical health T-scores also rose significantly (T0=44.8, T1=47.6, p<0.000) but no changes we found in the mental health T scores (T0=45.8, T1=46.3 p=0.709).
Conclusions Our results suggest that the NP-Led primary care model studied here likely constitutes an effective approach to improve primary care accessibility and quality.
Strengths and limitations
This study evaluates the impact of a primary care nurse-practitioner-led clinic model piloted in British Columbia (Canada) on patients’ health and care experience
The study rests on a pre-post survey without a control group therefore the differences observed could be caused by external factors
Data collection took place between 2020 and 2022, during the Covid-19 pandemic.
Only four NP-PCC clinics exist and participation in the survey was voluntary and uncompensated limiting the number of respondents
INTRODUCTION
The lack of timely access to primary care services has been a significant problem across Canada for decades. Nearly one in seven Canadians had reported not having a regular healthcare provider in 2019 (1) and, in 2020, 40% reported having an avoidable emergency department visit because of the lack of primary care options (2). Available indicators also suggest the situation has significantly worsened since the Covid-19 pandemic.
Increased reliance on non-physicians such as registered nurses and Nurse Practitioners (NPs) has long been documented as a way to improve primary care access (3, 4, 5, 6, 7, 8). In Canada, NPs are registered nurses with strong practice experience as well as a specialized master or doctorate level training. They have a broader scope of practice including the ability to autonomously diagnose, prescribe treatments and refer patients to test or specialist physicians.
Several models have been used to integrate NPs into primary care delivery. The model with the highest disruption potential is called the NP-Led Clinic (9). Those clinics have NPs as the pivot professional of a team providing the full scope of primary care services to a panel of patients (3, 10). The NP-led clinic model has consistently been found to be effective and efficient in meeting the primary care needs of a general population (6, 8, 11, 12, 13).
Despite the strength and abundance of evidence supporting the NP-Led model, its implementation in Canada has been limited and patchy. In 2020, in response to the widespread need for primary care and the successful advocacy from its NP association (14), British Columbia (BC), one of Canada’s 10 provinces, opened four pilot NP-led clinics. The BC model was named Nurse-Practitioner Primary Care Clinic (NP-PCC).
The aim of this study is to evaluate the impact of NP-PCCs on the health and patient reported experience of the first cohorts of rostered patients.
BACKGROUND
The pilot NP-PCCs are located in four cities ranging from core urban to peri-rural. The Axis clinic is in Surrey, Nexus is in Nanaimo, Health Care on Yates (HCOY) is in Victoria and Flowerstone is in Qualicum Beach. From the onset, the clinics were tasked with providing comprehensive primary care services to people from formally defined catchment areas who were without a primary care provider. The rostering goal for all clinics was set at 6800 patients after 3 years of operation by the Ministry of Health. Patients are formally rostered to the clinic as well as informally attached to a specific NP within the clinic. Due to very high demand, all four clinics used patient waitlists or random draws as a part of their attachment processes. On average, each clinic has the equivalent of six full-time NPs, one social worker, one mental health counsellor or clinician, two RNs, one clinic manager, and four Medical Office Assistants. Clinics are funded by the BC Ministry of Health and have a self-appointed board of directors and managerial autonomy (15).
DATA AND METHODS
Study design
The study relies on a quasi-experimental, longitudinal design based on a pre-and-post survey of patients receiving care in a NP-PCC. The pre-rostering survey (Time zero or T0) asked respondents to describe their care experience during the two years preceding being rostered to a NP-PCC as well as their current health. Approximately one year later, patients were asked to complete a second (Time one or T1) similar survey, this time focused on their care experience as an NP-PCC patient. The design was adapted from previous work done by the same team (10, 16, 17).
Participant Recruitment
For the T0 recruitment, patients newly rostered to one of the clinics were given an information sheet about the project by the clinic’s clerical staff or by their care provider. The information sheet included a link to a consent form and online survey instrument hosted on the University of Victoria Survey Monkey platform. The survey asked patients to provide their government-issued personal health number (PHN). The PHNs of patients who completed the T0 survey were shared with the clinics’ clerical staff who made a note in the clinics’ Electronic Health Record system. About a year later during a subsequent visits to the clinic, those patients were provided with an invitation to complete the T1 survey. Email invitations were also sent by the clinics to patients identified as having completed the T0 survey. Data collection periods varied slightly from site to site (see Figure 5 in Appendix) but, overall, the T0 period ran from November 2020 to January 2022 and the T1 period ran from December 2021 to December 2022. The study was approved by the University of Victoria Human Research Ethics Board (#20-0324)
Instruments
The patient reported experience from being rostered in a NP-PCC were measured according to five core dimensions of primary care experience: accessibility, continuity, comprehensiveness, responsiveness, and outcomes of care. Those dimensions have been extensively discussed and conceptualized in the literature (18, 19, 20). The definition of accessibility rests on Donabedian’s seminal work (21), as the fit between the structures of production, on one hand, and society’s needs and their geographic distribution, on the other. It is operationalized here as the patient’s perception that they can access healthcare services without undue constraints. Continuity is defined, based on the work of Haggerty et al. (22), as the fact that a patient is treated by the same professional or the same team over time (relational continuity) and that different services are harmoniously integrated with each other (management continuity). Comprehensiveness encompasses two dimensions that make up the scope of patient management: considering all of a patient’s needs and providing a complete basket of services (10, 23). Responsiveness is defined here as the convergence between the patients’ expectations regarding non-technical elements of the care and what the clinic offers in practice (20). Finally, what is described as outcomes of care relates to the patients’ perception that the care received had a positive impact on their health (24).
The five dimensions described above were measured using a survey instrument adapted from the one developed by Pineault et al. (24, 25) which was in turn an expanded version of two well-validated pre-existing instruments: the Primary Care Assessment Survey (26) and the Primary Care Assessment Tool (27). Details on the phrasing of the survey questions and computation of scores can be found elsewhere (17, 25). Each score is measured on a scale ranging from 0 to 10.
In addition to the patient-focused indicators described above, we also measured respondents’ general health status using the SF-12 Short-form Health Survey (Ware et al., 1996). The SF-12 instrument provides a physical health score and a mental health score, both measured on a scale ranging from 0 to 100 and calibrated at 50 for the average US adult population.
Methods
At the end of the data collection period, patients’ PHNs were used to individually match T0 and T1 survey answers. Survey answers that couldn’t be matched (missing or divergent PHN) were removed from the database. In the same way, missing data either at T0 or T1 for any given score led to the removal of those responses in the computation of mean scores and differences. Scores were computed in Microsoft Excel.
There were 437 usable T0 surveys and 254 matched and usable T1 surveys (see
INSERT Figure 5 in appendix). The representativeness of the survey respondents as compared to the population of the clinics’ catchment areas as well as to the clinics’ rosters was analyzed at the end of T0 data collection and results have been published elsewhere (28). Representativeness was generally good. At the end of the data collection period, we used descriptive and inferential statistics on socio-demographic information to assess the risk of attrition bias. For categorical data we used Chi square tests based on the actual T1 number of respondents per category and expected T1 numbers based on T0 proportions. Age was assessed both as a categorical variable based on 10-year age groups and by a t-test of the mean.
Primary outcomes were analyzed using paired t-tests of the mean (T1-T0) on each of the five scores related to the core dimensions of patients’ primary care experiences as well as for the mental and physical health scores. Computations were done in IBM SPSS (v26).
RESULTS
The overall completion rate of the T1 survey from T0 respondents was 58% (individual clinics rate varied from 44% to 72%). The Chi2 tests conducted showed no significant indication of an attrition bias in the core socio-demographic variables captured by the survey (See Table 1 in appendix)
The main finding from the longitudinal analysis of patients’ experience of care is that there was a large increase in the scores for all dimensions measured (see INSERT Figure 1 below). Patient perceptions of accessibility, continuity, comprehensiveness, responsiveness and that the care received made a difference on their health were significantly higher compared to the period preceding their rostering in the clinic. Probabilities from the paired t-test of the mean are below 0.000 for all five dimensions (see Table 2 in appendix)
There was some inter-site variability in pre-and post-scores but the overall trend of significant improvements remains (see INSERT Figure 2 below). All dimensions of patients’ primary care experiences vastly improved at all four sites. The pre-post differences for individual clinics are again significant (p<0.00) for all dimensions (see Table 2 in appendix).
The composite experience of care scores also aligns with individual survey questions focused on similar constructs. For example, the proportion of respondents reporting unmet needs (based on the question, “In the last year, did you feel the need to see a doctor or another health professional for a health problem, but didn’t see one”) sharply decreased between T0 and T1. The overall proportion of patients reporting unmet needs at T0 was 60% (ranging from 45% at Axis to 67% at HCOY). At T1 that proportion was down to 19% (ranging from 4% at Axis to 33% at HCOY). As the question is not specific to primary care, challenges in access to specialist physicians or other services in each area should be considered in the interpretation of those scores.
Regarding respondents’ health condition as measured by the SF-12 instruments, our results show an overall improvement in the physical health scores (rising from 44.8 at T0 to 47.6 at T1 p<0.000) and no change in the mental health score (from 45.8 at T0 to 46.3 at T1 p=0.709) (see INSERT Figure 3 below as well as Table 3 in appendix).
When analyzed on a site-by-site basis, the trend of a significant improvement in the SF-12 physical health score persists. The data shows a significant difference (at alpha 0.05) in pre-post physical health scores at three of the four clinics. Given the low number of responses (df ranging from 32 to 69), this trend is interesting.
When the matched T1-T0 differences in physical health scores are plotted in relation to the T1-T0 differences in mental health scores, it creates a scatter plot with four quadrants. The upper right quadrant contains respondents who reported both improved physical and mental health. The lower left quadrant contains respondents who reported both decreased physical and mental health. The two other quadrants correspond to people reporting improvements in one area and a decrease in the other. Overall, the figure shows that the improvement in the average physical health score is a product of both more people on the right on the graph (62% of respondents reported an improvement in their physical health) and the fact their average score difference is higher in absolute value (+7.6) as compared to respondents with a negative physical health score difference (−5.1). No such pattern exists for the mental health score differences (top versus bottom of the figure)
DISCUSSION
The main finding from the longitudinal analysis of patients’ experience of care is that there was a large increase in the scores for all dimensions measured after rostering in a NP-PCC. The improvements observed in patients’ experiences regarding primary care can be in part explained by the low pre-rostering (T0) baseline. Other studies that relied on the same instruments (17, 24) measured higher baseline or control-arm scores. However, the post-rostering scores (T1) measured here are similar to what was found for well-established medical clinics in those studies. Both the magnitude of the improvements measured here and the comparison of scores from the NP-PCC model as compared with the ones of large well-established interdisciplinary clinics found in other studies (17) suggest that NP-PCCs are an effective model of primary care delivery.
Given the short duration of the longitudinal follow-up conducted here, we were surprised to observe a significant improvement in the physical health as measured by the SF-12 instrument. Informal discussions with NPs working at the HCOY clinic suggest that the NP-PCCs rostered a large proportion of patients who had been left with unmet need for long periods of time. Our hypothesis is that providing accessible and comprehensive care to those patients might have been enough to cause the observed improvements in physical health. It could also be that, given recovery from mental health condition is usually longer, our study period was too short to catch improvements at that level.
More generally, it might be stressed that the NP-PCC model was, from the onset, based on the assumption those clinics would focus on rostering patients who do not have a regular provider. The survey data confirms most patients did fit that profile and that they faced low accessibility to care as well as generally low quality of care before joining the NP-PCC (28). The proportion of survey respondents reporting unmet needs at T0 (60%) is many times higher than what is observed in other populations. In 2022, 7.7 % of people in British Columbia and 7.2 Canada declared they didn’t get care when they had needs (29) while the European Union average for medical care was at 4.8% (30).
Limitations
This study has some limitations. First, it rests on a quasi-experimental design based on a pre-post survey without a control group. As such there is a risk that external factors explain the differences observed here. The most notable external factor is that the data collection took place between 2020 and 2022, during the Covid-19 pandemic. To control for some biases, the pre-rostering survey questions were edited to ask respondents about the care they received during the “last two years” therefore including one pre-pandemic year. This phrasing is likely to have overinflated T0 scores as compared to what they would have been if those patients had to seek care outside of the NP-PCCs at that time. It might also be worth mentioning that all available indicators suggest that the primary care delivery capacity in BC sharply declined throughout the data collection period (31). Both the phrasing of the T0 survey and the context during data collection are likely to have had a negative impact on T1 to T0 score differences making the magnitude of the differences observed here even more interesting. Finally, we want to mention that the limited number of respondents is explained mostly by the fact only four NP-PCC clinics exist and that participation in the survey was voluntary and uncompensated. Nevertheless, respondents were generally representative of the total clinic roster population (28).
CONCLUSION
Our findings show that offering people who were relying on low-continuity, low comprehensiveness services, like medical walk-in clinics, access to an NP-Led alternative dramatically improves their care experience. All dimensions of care are positively affected, and our results suggest that being followed in an NP-PCC could lead to gains in patients’ overall physical health. The patients’ primary care experience scores measured in our study favourably compare to the ones measured in well-established interdisciplinary and medical clinics.
Our findings are well aligned with the large body of evidence showing that NP-led primary care is equivalent or better than average available medical care (6, 8, 11, 32). Our study also shows that BC’s NP-PCC model does produce the results that were hoped for when the model was launched (15) and constitutes an effective approach to improve primary care accessibility and quality. As the first four NP-PCC were funded as pilots of a potential scaling up of the model, our results can play a role in future policy decisions to that effect.
This work was supported, in part, by the non-profit Nurses and Nurse Practitioners of British Columbia (NNPBC) association
Data Availability
The nature of the data and conditions set by the University of Victoria Human Research Ethics Board prevent the sharing of the raw data.
Competing interests
The authors report no competing interests. Katherine Bertoni is a locum practitioner in one of the clinics in which the study took place.
Authors contributions
DC, KB and AD designed the study. DC, KB and GR were involved in data collection. DC ran the computations and analyses and wrote the first draft. DC KB, AD and GR were involved in writing the final manuscript. The authors also want to thank Breanna Horne (Uvic), Ashleigh Swanson (Uvic), Shawna Glassel (NNPBC), Harjot Grewal (NNPBC), Lynn Guengerich (Health Care on Yates), Kari Jonker (Nexus), Lexi Grisdale (Axis) and Liz Gilmour (Flowerstone) for their support in this project.
Data sharing statement
The nature of the data and conditions set by the University of Victoria Human Research Ethics Board prevent the sharing of the raw data.
APPENDIX
Footnotes
damien1{at}uvic.ca, kbertoni{at}uvic.ca, arnaud.duhoux{at}umontreal.ca, gurprit{at}uvic.ca