Medical students’ and educators’ opinions of teleconsultation in practice and undergraduate education: a UK-based mixed-methods study
=========================================================================================================================================

* Lisa-Christin Wetzlmair
* Andrew O’Malley
* Veronica O’Carroll

## Abstract

**Introduction** As information and communication technology continues to shape the healthcare landscape, future medical practitioners need to be equipped with skills and competencies that ensure safe, high-quality, and person-centred healthcare in a digitised healthcare system. This study investigated undergraduate medical students’ and medical educators’ opinions of teleconsultation practice in general and their opinions of teleconsultation education.

**Methods** This study used a cross-sectional, mixed-methods approach, utilising the additional coverage design to sequence and integrate qualitative and quantitative data. An online questionnaire was sent out to all medical schools in the UK, inviting undergraduate medical students and medical educators to participate. Questionnaire participants were given the opportunity to take part in a qualitative semi-structured interview. Descriptive and correlation analyses and a thematic analysis were conducted.

**Results** A total of 248 participants completed the questionnaire and 23 interviews were conducted. Saving time and the reduced risks of transmitting infectious diseases were identified as common advantages of using teleconsultation. However, concerns about confidentiality and accessibility to services were expressed by students and educators. Eight themes were identified from the thematic analysis. The themes relevant to teleconsultation practice were (1) The benefit of teleconsultations, (2) A second-best option, (3) Patient choice, (4) Teleconsultations differ from in-person interactions, and (5) Impact on the healthcare system. The themes relevant to teleconsultation education were (6) Considerations and reflections on required skills, (7) Learning and teaching content, and (8) The future of teleconsultation education.

**Discussion** The results of this study have implications for both medical practice and education. Patient confidentiality, safety, respecting patients’ preferences, and accessibility are important considerations for implementing teleconsultations in practice. Education should focus on assessing the appropriateness of teleconsultations, offering accessible and equal care, and developing skills for effective communication and clinical reasoning. High-quality teleconsultation education can influence teleconsultation practice.

Keywords
*   Teleconsultation
*   telehealth
*   digital health
*   medical education
*   mixed methods

## Introduction

Telehealth uses technology synchronously or asynchronously to deliver, manage, coordinate, and evaluate health and well-being services remotely (1–4). The utilised technologies cover a range of methods such as real-time remote encounters with healthcare professionals over email, telephone, or video-call, or the access to educational material or electronically stored healthcare information (for example in electronic health records) (5). The integration of telehealth in healthcare practice and education can revolutionise the way future and present healthcare professionals learn, practice, and ultimately deliver patient care. The use of telehealth services will continue to shape the global healthcare landscape (6,7) as healthcare systems worldwide face challenges such as workforce shortage and the increased complexity of long-term conditions and multimorbidity. The COVID-19 pandemic put additional stress on healthcare systems (8) and prompted a surge in telehealth services offered to patients and the publication of telehealth policies and guidelines between 2020 and 2022 (9–12). Telehealth could mitigate against, for example, limited human resources, equipment, and health facilities by accelerating access to information for healthcare professionals and increasing accessibility to services for various patient and population groups (13,14).

One essential sub-service of telehealth is teleconsultation, which can increase patients’ accessibility to healthcare and reportedly allows the extension of healthcare services to remote and rural areas (1,15,16). Teleconsultations are defined as direct, synchronous, or asynchronous consultations between healthcare professionals and patients using communication technologies such as phone or video calls, or emails (17). Evidence-based evaluations have shown similar patient outcomes for in-person and teleconsultations in primary care and mental health services (18). The use of teleconsultations has been shown to improve patient management, decrease referral rates and the number of medical procedures, and shorten diagnostic times (17). Although patients have reported benefits of and satisfaction with teleconsultations (19,20), concerns have also been highlighted from the practitioners’ perspectives such as increased workload based on an increase in demand for teleconsultations (20,21). Additionally, in some instances, dissatisfaction with in-person consultations following an initial remote consultation has been reported from both patient and practitioner perspectives. Evaluations of teleconsultation platforms in the UK showed that healthcare professionals prefer to use teleconsultations only “for routine follow-up of chronic, stable conditions” over acute conditions (20,22). Nevertheless, teleconsultations are often utilised in acute care settings such as out-of-hours services or clinical triaging for emergencies (23).

During the pandemic, Universities utilised teleconsultation platforms to ensure that medical education continued during governmental lockdowns (24–26). This was implemented out of necessity to ensure that students continued to have patient contact. The volume of teleconsultation experience is likely to have been a stark contrast to the limited teleconsultation education featured in health and social care programmes prior to the pandemic, as reported in a systematic review by Wetzlmair et al. (27). This review highlighted that good quality teleconsultation education is vital for encouraging effective teleconsultation practice. Previous studies have reported that teleconsultation education at the pre-registration or pre-qualification level ultimately ensures safe and high-quality remote healthcare for patients (21,28), as education initiatives can ensure that medical education meets the requirements and needs of the health and social care sector (29,30). However, only tailored education increases the likelihood of successful implementation of teleconsultation services and benefits practitioners’ knowledge, skills, and attitudes (31,32).

Thomas et al. (33) identified education and training of both future and current workforce as key to ensuring sustainable and safe teleconsultations. A skilled and competent healthcare workforce should not only use technology competently and effectively, but should also be able to assess potential implications of the used technology on patients’ safety, access to health and social care, and ethical and legal consequences (34–37). As the COVID-19 pandemic accelerated the use of teleconsultations across the health and social care sector (38,39) and the use of teleconsultation continues to expand, the education of present and future practitioners will become crucial. However, with limited educational frameworks and standardised teleconsultation integration into healthcare curricula (40), healthcare professionals may not be provided with the opportunities to build the competencies and confidence they need, to be able to work in a digitised healthcare system (29,33,41). As teleconsultations are only sporadically integrated into most medical curricula (27), medical education and research need to investigate factors that inform the development and ultimately implementation of teleconsultation education.

This paper focuses on the findings related to two objectives of a larger study undertaken as part of a PhD. The PhD study aimed to investigate the factors that inform the development of teleconsultation education in undergraduate medical education in the UK. The two objectives focused on in this paper are: 

1.  to determine the opinions of undergraduate medical students and medical educators on teleconsultation applied in healthcare practice,

2.  to determine the opinions of undergraduate medical students and medical educators on teleconsultation utilised in educational settings.

## Methods

This study used a cross-sectional, convergent mixed-methods study design (Figure 1) where both quantitative and qualitative data collection methods occurred simultaneously (42–44) to investigate undergraduate medical students’ and medical educators’ opinions of teleconsultation practice and education in the UK. This mixed methods model enables the researcher to address different aspects of the research questions and compare, contrast, and correlate results to provide more comprehensive information for the interpretation of the results (45,46). The results from this study were analysed separately and integrated during data interpretation.

![Figure 1:](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2024/04/01/2024.03.28.24305056/F1.medium.gif)

[Figure 1:](http://medrxiv.org/content/early/2024/04/01/2024.03.28.24305056/F1)

Figure 1: 
Illustration of a convergent mixed-methods design with qualitative priority and quantitative input, based on (45–48). *Delay due to the optional sign-up after responding to the questionnaire.

Both research design and research methods followed a mixture of constructivist and post-positivist approaches, with more emphasis put on the constructivist paradigm. A-priori decisions on sequencing priorities and assumptions on how to integrate quantitative and qualitative methods were made and influenced by the research paradigm.

### Development and administration of the research instruments

The development of the questionnaire was informed by the Technology Acceptance Model (TAM) (49) and findings from a systematic literature review related to commonly reported teaching methods (27). Items were assessed for their validity and reliability (50) by three independent researchers (VOC, AOM, LCW) and were subject to a test-retest approach involving five medical students and five medical educators. The questionnaire was administered using the online software Qualtrics (51). The final version of the questionnaire (S1 Appendix) comprised a total of 29 questions. Display logic was introduced as not all questions were equally relevant to all participants. Most questions were closed, requiring participants to indicate a response from single or multiple-choice options and five-point Likert scale responses. These scales assessed the strength of agreement related to perceived usefulness and perceived ease of use of, attitudes towards, and difficulties in the use of teleconsultations. The final questions asked participants to rank the perceived advantages and disadvantages for patients and healthcare professionals using teleconsultations. The ranking items derived from the literature and are summarised in the S2 Appendix. The questionnaire ended with an open-ended question, enabling participants to provide further comments on the perceived advantages and disadvantages of teleconsultations in clinical practice and medical education.

A topic guide (S3 Appendix) was used to guide the discussion in the semi-structured interviews. The topic guide was based on the research objectives, specifically the objective of gaining a more in-depth understanding of personal opinions of teleconsultation practice and medical education at UK medical schools. The interviews were conducted remotely by one researcher (LCW) utilising the software Microsoft Teams (52) between the 14th of June 2021 and the 16th of March 2022. A real-time analysis of the first ten interviews ensured that the interviewer (LCW) adhered to the interview guide (53).

### Recruitment and sampling

Medical educators working in a clinical and/or academic setting in the UK and undergraduate medical students from all year groups studying at a UK medical school were eligible to participate in the study. The online questionnaire was distributed to medical students studying in university in the UK and to medical educators working within academic and/or healthcare settings in the UK between the 19th of May 2021 to the 13th of March 2022. After completing the questionnaire, participants were given the option to participate in a semi-structured interview. Written consent was obtained on the first page of the questionnaire. Interview participants consented to the interview via email prior to the interview.

The recruitment techniques for the quantitative aspect of the study were cluster, convenience, and snowball sampling (54). For the initial sampling strategy, a cluster of all medical schools in the UK (to date 2020, n=42) was created and a participant recruitment letter was sent to the Heads of Schools asking them to distribute the link to the questionnaire to their undergraduate medical students and medical educators affiliated with the respective school. In addition, separate emails were sent to the medical directors of Wales (n=1) and Northern Ireland (n=5) and to the Directors of Medical Education within Scottish health boards (n=15). Convenience sampling referred to the sampling strategy utilising the social media platform X (formerly Twitter) to recruit contacts known to the research team. Those contacts (“gatekeeper”, n=7) were clinicians with involvement in undergraduate medical education across the UK. A snowball sampling technique was applied by encouraging gatekeepers and interview participants to distribute the questionnaire to their study and work colleagues. The qualitative sampling technique was based on self-selection. All study participants who participated in the online questionnaire were offered the option to participate in a semi-structured interview.

Data collection was initiated after the initial ethics approval received from the School of Medicine at the University of St Andrews (approval code MD15263). Two ethical amendments were approved to reflect changes to the initial recruitment strategy. Additional approval and permission to undertake the research was provided by the UK Medical Schools Council (MSC), and individual UK medical schools upon request.

### Data analysis

The quantitative data analysis was based on descriptive and correlation analyses. To align with the objectives of this paper, only a subsection of the questionnaire items was included in the quantitative analysis. These items included: demographical questions, ranking tasks regarding advantages and disadvantages perceived from a healthcare professional and patient perspective, and opinions of the inclusion of teleconsultation education within the UK medical curriculum. Participant characteristics were reported by measure of central tendency (mean, standard deviation) (55). The Spearman-rank correlation method was applied to the categorical variables (56) and is subsequently reported with the coefficient *ρ* (rho). The ranking items were analysed using frequencies and utilised a weighted average of the responses to account for the weight (i.e. importance) of the responses (57). In a two-step approach, a rank score (RS) was calculated by multiplying the number of responses of each item with the “weight” of the answer (e.g. 7 is the most important advantage, 1 is the least important advantage). This RS was then divided by the total number of responses. The quantitative data analysis was performed utilising the Statistical Package for Social Sciences (SPSS), version 26 (58), and Microsoft Excel (59).

The qualitative data included the verbatim transcribed interview data and responses to the open-ended items in the questionnaire. The data was analysed thematically, utilising an inductive, organic analysis, mainly guided by the school of reflexive thematic analysis (53). After an initial coding phase, one author (LCW) named, defined, and described codes in a codebook. This codebook was shared with two other researchers (VOC, AOM) who each coded six different transcripts independently to ensure transparency, reliability, and validity (53,60). An inter-coder-reliability was calculated and a threshold of 80% agreement was defined. Agreement rates above 80% were not subject to further discussions. Rates below 80% were not accepted, hence differences in coding were investigated and discussed between the researchers until a consensus was found. Data analysis of the interview transcripts and the open-ended questionnaire item was conducted in NVivo (61).

## Results

### Quantitative Analysis

A total of 240 questionnaire responses were considered for statistical analysis. Participant characteristics are summarised in Table 1.

View this table:
[Table 1:](http://medrxiv.org/content/early/2024/04/01/2024.03.28.24305056/T1)

Table 1: Participants’ characteristics stratified by their role in medical education (i.e. medical students and medical educators)

Regarding the level of education, half the students reported that they had completed education at a high school or college level (55.6%). Forty-seven reported having a Bachelor’s degree (30.7%) and 13 had a Master’s degree (8.5%). Three participants had a PhD (2.0%). Half of the students were in their first (n=36, 23.5%) or second year (n=38, 24.8%) of medical education, 23 students (15.0%) and 31 students (20.3%) were in their third or fourth year, respectively. Year five and year six were attended by eleven (7.2%) and six (3.9%) students, respectively. No student was intercalating when they completed the questionnaire. Amongst the medical educators, most participants had a Master’s degree (n=27, 31.0%). A further 17 (19.5%) and 10 (11.5%) had a PhD or MD, respectively. Medical educators working in both settings (n=34) reported that they spent, on average, more hours in a healthcare setting (*x*=23.89, sd=15.42) than in an academic setting (*x*=15.07, sd=12.24).

Participants’ opinions of the advantages and disadvantages of teleconsultations are summarised in Table 2 and Table 3, respectively. The advantages reported by healthcare professionals were ranked by 155 participants. The two most important advantages included the reduction of transmitting contagious diseases (*x*=5.37, RS=833) and easier accessibility to remote populations (*x*=5.23, RS=810). The avoidance of unnecessary medical procedures was the only item ranked on average between second to last or last (*x*=1.89, RS=293). Disadvantages were ranked by 153 participants. The results were more bimodal than the perceived advantages: half of the items scored on average between first and second place: Experiencing technical problems and concerns about confidentiality were both rated the same and perceived as important disadvantages (*x*=4.22, RS=645). The most important disadvantage was the limitations in physical examination (*x*=4.99, RS=764). The perception that delivering difficult or upsetting news is harder was perceived as the last important (*x*=2.32, RS=355) disadvantage of teleconsultations. Advantages from a patient perspective were ranked by 146 participants. The perceived time-saving component was the most important advantage (*x*=5.86, RS=856). Similar to healthcare professionals, the avoidance of unnecessary medical procedures was the least important perceived advantage (*x*=1.72, RS=251). A total of 150 participants ranked the disadvantage from a patient perspective. Access to technology (*x*=4.33, RS=650) and potential risks for miscommunication due to the limited use of non-verbal cues (*x*=4.25, RS=637) were the most important disadvantages. Unlike the ranking for healthcare professionals, from a patient perspective, confidentiality concerns were perceived as the least important (*x* =1.82, RS=273) disadvantage of teleconsultations.

View this table:
[Table 2:](http://medrxiv.org/content/early/2024/04/01/2024.03.28.24305056/T2)

Table 2: Results of the ranking item regarding the perceived advantages of teleconsultations from the perspective of healthcare professionals and patients. Scale from 1 (least important) to 7 (most important).

View this table:
[Table 3:](http://medrxiv.org/content/early/2024/04/01/2024.03.28.24305056/T3)

Table 3: Results of the ranking item regarding the perceived disadvantages of teleconsultations from the perspective of healthcare professionals and patients. Scale from 1 (least important) to 6 (most important).

Medical students’ opinions of the implementation of teleconsultation in the medical curriculum are significantly positively correlated with previous exposure in clinical placements (*ρ* =.205; p=.038); however, previous exposure in academic learning environments is insignificantly negatively correlated (*ρ* = -.113; p=.225) with positive attitudes towards integrating teleconsultation in undergraduate medical education. There was no significant correlation between medical educators offering teleconsultations in a healthcare setting (*ρ* = -.030; p=.825) and teaching teleconsultation in an academic setting (*ρ* =.257; p=.051) and their inclination to implement teleconsultation in the medical curriculum.

### Qualitative Analysis

A total of 23 interviews were conducted and analysed. 13 interview participants were medical students and ten were medical educators. Nine students (69.23%) and six educators (60.00%) identified as female. Participants were mainly geographically located in England (n=12, 52.17%) and Scotland (n=11, 47.38%). A total of eight themes were identified inductively from the thematic analysis. Five themes were related to opinions of teleconsultation in clinical practice and three themes were related to teleconsultation education (Table 4).

View this table:
[Table 4:](http://medrxiv.org/content/early/2024/04/01/2024.03.28.24305056/T4)

Table 4: Research objectives and identified themes.

### Opinions of teleconsultation practice

#### Theme 1: The benefits of teleconsultations

As illustrated in the quotes below, participants highlighted the personal and professional benefits of a hybrid approach to teleconsultations, comprising of a mix of in-person and remote:

> *I think probably the best way to manage it [telehealth] is to do a mixture of both that suits the parent and the child, but it should be seen as an adjunct, I think, rather than a sort of primary mode of assessment and intervention. – Interview #2 (Medical Educator, both settings)*
> 
> *… it’s tricky and you gotta have the right sort of patient as well. I mean, some people just don’t really bother open-up on the phone (…) at the rural practice I think it’s got less of a place than in inner city because you’ve got internet problems, phone signal isn’t there, infrastructure issues … as well. – Interview #13(Student)*
> 
> *… so now we mostly do it [teleconsultation] based on what is best for the patients (…) so depends a lot on urgency and severity of illness, whether or not it’s something that can be managed on the phone or whether or not you really need to go and speak to them because they might not want to speak to you on the phone as well so whatever the patient preferences – Interview #6 (Medical Educator, both settings)*

Teleconsultations were perceived to enable practitioners to work remotely and provide flexibility. Seeing patients in their personal environment, including caregivers and family members in the consultation, and observing the interactions between caregivers and patients might reveal essential information, which can contribute to a high-quality, patient-centred healthcare management plan:

> *… but it you know what I mean I’d rather have Mary at home, the nurse doing that for them … rather than drag Mary to my clinic. …. I did get a fair few patients with dementia as well, so I see them with their carer so that they are in their home environment, I’m not removing them away from it so it becomes more safe and secure for them … – Interview #3 (Medical Educator, healthcare setting)*

#### Theme 2: Patient choice

Participants discussed the need for and benefit of patient choice in relation to using either face-to-face or remote consulting modalities to suit their needs:

> *I mean the other thing we teach students as well is patient preference and patient choice, which is why I think from the pandemic we’re not gonna go back to where we were. – Interview #2 (Medical Educator, both settings)*
> 
> *… giving the patient that autonomy to choose and be more involved in their healthcare, I think, is one of the main priorities ‘cause everyone is all about the patient-centred care – Interview #14 (Student)*

The general understanding of patient choices was based on the opinions that patients preferred these options to be flexible in managing their time and patients habituated to remote consultations during the COVID-19 pandemic:

> *What patients have realized is that they can fill in a e-consultation form to get their message across, to give to a practice to then slot into an appointment slot, whatever that may look like and of course they can request can I have a face to face or can I kind of have a teleconsultation because the default always used to be face-to-face. – Interview #3 (Medical Educator, both settings)*

#### Theme 3: A second-best option

Patient safety and accessibility were identified as some of the main drivers for perceiving teleconsultation services as the second-best option. Cognitive and physical impairment and potentially differing levels of digital literacy amongst patients were considered as factors that may play a part in limiting accessibility to teleconsultations:

> *Teleconsultations can put patient safety at risk from a global perspective. – Questionnaire response #49 (Medical Educator, both settings)*
> 
> *The difficulty of actually getting hold of a healthcare professional, particularly if elderly or not technologically able. Many people feel like their doctor can just hide away and not be contacted. – Questionnaire response #195 (Student)*
> 
> *… probably the hardest thing is actually the patient side of the spectrum, really … if you’re dealing with a lot of older people or people who are not digitally literate, it makes things obviously also a lot harder, or if there’s like learning difficulties at the other side or like aids like hearing aids you can’t really account for that … – Interview #8 (Student)*

#### Theme 4: Teleconsultations differ from in-person interactions

Students as well as educators felt that that not all types of consultations with patients such as breaking bad news, discussing sensitive information, or where a patient needed to be examined were suitable for teleconsultations. Deciding before a consultation whether a teleconsultation would be an appropriate method to use over an in-person consultation was deemed important:

> *I think there are certain things that you don’t want to discuss over the phone. So, difficult news or very difficult conversations can be challenging over the phone and again, if someone is trying to bring up a difficult subject, particularly around mental health, you need to a lot of your assessment will be what they are and what they look like, and it feels quite impersonal having these discussions on these topics without sitting face-to-face. – Interview #20, (Student)*
> 
> *And I think being clear what you can and can’t do on a videoconsultation. We can do reasonably well with a history … examination of the patient I think is quite limited. And so that will affect the type of consultation … Some things will be suitable for a videoconsultation some things will not. – Interview #1 (Medical Educator, healthcare setting)*
> 
> *… also, that we cannot physically examine the patient and looking at a picture and looking at something in person is quite different … – Interview #7 (Student)*
> 
> *Without the ability to perform physical examinations, care is less safe. Pts [patients] will be exposed to more harmful scans and tests to make up for the lack of physical examination – Questionnaire response #10 (Medical Educator, both settings)*

Medical students perceived that it was easier to build rapport and interpret and acknowledge non-verbal communication cues during in-person consultations compared to teleconsultations:

> *… it’s a lot easier if you’re sitting with someone in a room to make them comfortable and relaxed, whereas if you’re over Teams I don’t know there’s still that kind of barrier, it’s not connected really. … – Interview #19, (Student)*
> 
> *difficult to gauge mood, etc, difficult to gauge social circumstances, difficult to gauge gait for e.g. as part of a quick assessment of a patient. – Questionnaire response #62 (Student)*

Educators with experience in academic and clinical learning environments highlighted the differences in information the practitioners can gather and professional relationships during remote consultations compared to in-person appointments:

> *… they [clinicians, students] need to have an awareness that they’re not getting the information or the whole picture that they probably need … really a crucial piece of information might be missing. – Interview #2, (Medical Educator, both settings)*
> 
> *Doctor-patient relationship is strengthened by face-to-face meetings. – Questionnaire response #35 (Medical Educator, both settings)*
> 
> *I think students (…) can find that quite disruptive because they’re already struggling a bit sometimes to get into the flow, and they’re really focusing and trying their best to build a rapport and if their phone suddenly goes dead … I think it throws them a bit, just like it throws me a bit, but I’ve got a bit more used to it. – Interview #6 (Medical Educator, both settings)*

#### Theme 5: Impact on the healthcare system

Participants mainly perceived that teleconsultations did not save time in clinical practice and suggested that it might even introduce more complex working environments where hybrid appointments might duplicate work or increase the overall workload for practitioners:

A lot of patients end up needing to come in anyway for tests/examinations, so telecommunication prior to this seems like extra time used – *Questionnaire response #71 (Student)*

So, it’s very important not to conflate the ideas that more telephone consultation equals I can see more patients because that’s totally wrong and at its worst telephone consultations done badly leads to more work because the patients are left unfulfilled. – Interview #3 (Medical Educator, both settings)

The ones [patients] that I send in to be seen face-to-face are the ones that should have been face-to-face in the first place so one of the things about telemedicine is that you select the right patients for your remote consultations. – Interview #4 (Medical Educator, healthcare setting)

Students perceived that teleconsultations are most likely to be successful if the coordination between medical practitioners and other members of the interprofessional team is effective. Students mostly referred to the role of nurses or receptionists in coordinating and triaging patients and the interprofessional collaboration between specialist units and the patients’ GPs:

I mean, triage systems run pretty well when they’re nurse lead, I think, and sort of other staff, I don’t think putting another burden onto GP is necessarily the way to do it because you’ve got them out there so that resource of a kind of other staff that are just to be able to do it most of the time … – Interview #13 (Student)

I feel like there’ll be more training required so like see receptionists who take the GPs calls. I think they should have a level of basic triage of like, right, don’t even bother having the online thing, come straight in, the doctor will see you, and I think that would be a very interesting point. Meanwhile, when I was at the GPs, it was very much up to the GP … - Interview #18 (Student)

I feel like a collaboration with local GPs for this kind of service [routine follow-ups] would make the online consult service much more efficient. – Questionnaire response #233 (Student)

### Opinions of teleconsultation education in undergraduate medical education

#### Theme 6: Considerations and reflections on required skills

Interview participants had varying opinions of whether teleconsultations should be utilised in undergraduate medical education. Participants disagreed on whether medical students and practitioners require different skills and competencies for teleconsultations than for in-person consultations. Some participants stressed that teleconsultation skills and competencies can be acquired regardless of dedicated education:

> *… in a way, I’m not sure how much training is needed or whether it’s more that kind of experiential learning of just doing it and seeing what it’s like. – Interview #4 (Medical Educator, healthcare setting)*
> 
> *… it [teleconsultation training] would be helpful, definitely but then I don’t know I feel like there’s so many other like priorities, that it’s something that you end up just picking up you know … – Interview #5 (Student)*
> 
> *If I can pick it [teleconsultation] up, I’m sure the younger generation as they come through (…) they might not even need specific training because they will just adapt quite easily. – Interview #3 (Medical Educator, healthcare setting)*

Other participants, however, saw the utilisation of teleconsultation education as a requirement for medical practitioners to navigate a digitised healthcare system:

> *… I think it’s actually a different skill (…) I’m just using the same history-taking style … that I start using in person, but it doesn’t always necessarily work quite as well I don’t think. – Interview #13 (Student)*
> 
> *I think there probably is specific work to be done about developing the telecare telehealth stuff as a skill in its own right, I am certain that the clinicians who do best on a ward round and that kind of thing are not always the same ones who are best doing remote consultations. – Interview #7 (Medical Educator, both settings)*

#### Theme 7: Learning and teaching content

Medical educators in favour of teleconsultation education highlighted specific learning and teaching content such as the opportunities and limitations of high-quality, safe, patient-centred telehealth. It was highlighted that these discussions should cover the limited possibilities for physical examination, potential technical difficulties, and confidentiality concerns.

> *The preparation for teleconsultations is significant. Learning about the ‘red flags’ /triggers for organising a follow-up F2F [face-to-face] appointment is still ongoing and I think that I am not expert at that yet. – Questionnaire response #70 (Medical Educator, both settings)*
> 
> *So, for example, somebody tries to show you a rash on a screen that is a bit blurry or pixelated and we just couldn’t see anything at all. So, I think it’s important that people [clinicians, students] don’t feel pushed into making a decision on the basis of inadequate information I think that-that can be quite dangerous. – Interview #1 (Medical Educator, healthcare setting)*

Students discussed that it would be essential to be taught how to ensure professionalism and patient confidentiality:

> *I think more so like confidentiality issues and so … if you leave a voicemail like can you say who’s calling and like from where that kind of thing, so, I think it would be more issues about confidentiality … – Interview #15 (Student)*

Furthermore, both students and educators mentioned that teleconsultation education should comprise appropriate methods for documenting remote encounters with patients and learning how to navigate the virtual consultation environment:

> *Documentation, I think, making sure that people you know are taking appropriate clinical notes so you know you need to record this consultation in exactly the same way as you would normally … What happens if that [connection] breaks down, what we gonna do. We can’t see things properly, so we have a backup telephone number … so for example in Near Me, the patient’s telephone number comes up on the screen, so you do have a note of it but if you lose the connection, you lose the telephone number. – Interview #1 (Medical Educator, healthcare setting)*
> 
> *Asking if the patient can see us, can hear us alright … these are basic things that we may not necessarily ask the patient if they’re in person, but you would need to ask that if you’re talking to them virtually, of course. – Interview #7 (Student)*

#### Theme 8: The future of teleconsultation education

The general opinion was that teleconsultations would continue to be an integral part of healthcare. It was highlighted that learning should take place in both academic and clinical learning environments. Online placements should be offered to students, allowing them to actively practice remote consultations as opposed to observing clinicians consulting with patients remotely:

> *(…) they’ve [students] maybe done some simulations in their universities first and they thought through the fact that they weren’t going to be able to avoid telecommunication, then I think they might come with more informed questions about how to develop their practice … so, I think it’s interesting to think about how they could come better prepared. – Interview #2 (Medical Educator, healthcare setting)*
> 
> *If I had the ability to include within my clinic the time to allow a student to make the phone call initially to then be able to discuss the case with them and call the family back or to take over the consultation to make the plan. I think it would give the students a real-life rather than virtual learning experience in terms of remote consultation. – Interview #2 (Medical Educator, healthcare setting)*

Interview responses indicated that teleconsultation education in both academic and clinical learning environments faces challenges. Whilst the medical curriculum is content-rich and adding teleconsultation education is not necessarily perceived as a priority by students, clinical placements that focus solely on teleconsultations might not be as educational for students and take away from other clinical skills:

> *It [teleconsultation education] would be helpful, definitely but then I don’t know I feel like there’s so many other like priorities, that it’s something that you end up just picking up you know … I think they probably just don’t have enough time to squeeze it in, and it’s considered low priority. – Interview #5 (Student)*
> 
> *But if you’re watching somebody else do teleconsultation … it’s really hard to concentrate on … you’re watching someone on the phone, so dull to watch, but doing it is completely different as like … you’re actually doing it, so it’s a lot more interesting. – Interview #13 (Student)*
> 
> *All virtual consultations limits face to face clinical examination skills learning for students. – Questionnaire Response #15 (Medical Educator, healthcare setting)*

## Discussion

This mixed-methods study investigated the opinions of teleconsultation in clinical practice and the opinions of utilising teleconsultation in undergraduate medical education within the UK. Medical students and medical educators had diverse opinions of the use of teleconsultation in practice. Whilst remote consultations might allow patients to be flexible and ensure more person-centred care, a large percentage of participants were critical of the advantages of teleconsultation in terms of offering safe and high-quality care, building rapport, ensuring equal and easy access to healthcare, saving time, and being appropriate to all patients’ health and social care needs. Even though some participants questioned the extent of education needed to acquire skills and competencies for teleconsultations, most participants highlighted the need for some teleconsultation education within the academic and clinical setting.

### Patient confidentiality, safety, and accessibility

The findings of this mixed-methods study contribute to the body of evidence hinting at mixed opinions of telehealth in medical practice regarding patient confidentiality, patient safety, and accessibility to services (62–65). This paper specifically investigated the opinions of medical students and medical educators in both clinical and academic practice. The results indicated that teleconsultations increase concerns about patients’ confidentiality, even though the benefits of seeing patients in their home environment and increasing access to care have been highlighted as an advantage in both quantitative and qualitative data.

Newly qualified doctors are required to communicate with patients “respecting confidentiality and maintaining professional standards of behaviour” (66). Living arrangements can introduce additional barriers to accessing telehealth services as privacy concerns (e.g. no private space for a teleconsultation) might impose challenges, particularly for people of lower socioeconomic status (67). The potential risk of being overheard might distract patients and decrease the opportunities to share essential, sensitive information (42,68). The findings in this study showed that participants’ opinions of implementing teleconsultation education into medical education are ambiguous. As concerns around confidentiality were a key factor shaping those opinions, the literature discusses telehealth competencies, which acknowledge the importance of incorporating methods of ensuring patient confidentiality in teleconsultations into medical curricula (35–37). Ethical medical practice in ensuring the patients’ confidentiality and privacy is applicable to both in-person and telehealth practices; however, specific considerations in remote consultations must be made (35,36). Experiences from clinical practice recommended discussions around ensuring privacy on patient, healthcare professional, clinical, and policy levels (69). Recommendations such as encouraging professionals and patients to use headphones, seeking a private room whenever possible, avoiding recording the consultations (69), and the use of platforms that comply with national regulations (70) are probably most appropriate for the UK undergraduate medical education context.

Within this study, patient-specific factors such as competencies in the use of technology, access to technologies, and digital literacy were perceived as important factors to consider in both clinical practice and medical education. These findings are in line with reports across telehealth literature which advocate for assessing patient factors to improve patient safety and satisfaction when engaging in remote consultations (42,64,65,71,72). Practitioners need to assess the patients’ competencies and confidence levels in using technologies for remote consultations to successfully tailor healthcare to the patients’ needs (73,74). In addition, the effective, safe use of teleconsultations is also dependent digital literacy (42,71,72). Digital literacy refers to the skills required to retrieve and apply healthcare information and knowledge utilising technology (75). Patients should have these abilities and the resources to engage in remote consultations. Healthcare professionals need a profound understanding of digital literacy and its importance for equal access to healthcare (76) to adequately address their patients’ needs.

Ultimately, the use of teleconsultations should neither jeopardise patient safety and quality of care nor increase inequalities in healthcare for vulnerable population groups (73,74), as teleconsultations will gain importance around global healthcare systems. Initiatives that foster the population’s digital skills and ultimately their ability to understand, retrieve, and act on information given in a remote consultation are essential to ensure an ongoing safe healthcare delivery in a digitised healthcare setting (77). On a policy level, awareness should be raised that access to adequate and affordable technology is an important social determinant of health, influencing most aspects defined by Dahlgren and Whitehead (78).

### Clinical appropriateness and workflow

Study results indicate that teleconsultations might save time for patients and reduce risks of transmitting infectious diseases. However, study participants remarked on the need for assessing the appropriateness of teleconsultations over in-person interactions. Previous authors have highlighted the need for assessing “patient and practice readiness and impact” (35) and “patient safety and appropriateness use of telehealth” (36). Undergraduate education should therefore prepare students to be able to make this assessment of whether a teleconsultation is clinically appropriate or not. This includes but is not limited to evaluating the benefits and risks of consulting with patients remotely and understanding clinical indicators for transferring remote interactions to in-person consultations (35). Providing high-quality, safe, effective, and efficient care utilising remote consultations requires medical practitioners to correctly recognise when patients might be in urgent need of emergency medical treatment (79). Undergraduate medical education could help students develop these skills by increasing their awareness of different decision-making and clinical reasoning processes required during teleconsultations.

Additionally, the clinical environment should be ready for teleconsultations. As both the interviews and questionnaire responses indicated, introducing teleconsultations might increase the complexity of existing clinical workflows. Farr et al. (20) evaluated the implementation of a remote consultation system and found that medical practitioners felt frustrated by the duplication of work and additional time requirements when patients need to be seen in person following a remote consultation. Reportedly, the seamless integration of hybrid working arrangements that entail switching between in-person and remote consultations in clinical practice often constitutes a barrier and reduces uptake rates of teleconsultations (80). And even though participants in this study perceived the closeness to the patients’ living environments to be benefitting a person-centred approach, limitations in building rapport and communicating effectively and safely using technology were stressed as caveats or disadvantages. If those concerns remain unmet in medical education, patients’ safety can be negatively impacted (81). Teleconsultation education should therefore ensure that students are skilled in taking a history and adapting their assessment skills where necessary and possible (82). This could be achieved through simulated teleconsultations or more opportunities for students to practice teleconsultations in clinical settings (81,82).

### Implications for medical education

The results indicated that teleconsultation should be included in medical education, but the implementation requires careful consideration of learning content, context, and outcomes. Quantitative and qualitative data have shown different results: As students who were exposed to teleconsultations in clinical practice were more likely to favour the implementation of teleconsultation education into medical education, medical practitioners are on average questioning the need to formally implement teleconsultation education. In the interviews, students were more sceptical about including additional content into already dense curricula. Nevertheless, the literature supports the assumption that training the workforce is fundamental to ensure safe, effective teleconsultations and to sustainably integrate remote consultations in hybrid clinical workflows (33,42). Considering the most likely hybrid future of healthcare delivery, the medical curricula must imbed teleconsultation skills in medical education. The findings from this study could serve as helpful guidance for medical educators to consider the implementation of teleconsultation.

The content of teleconsultation education is dependent on the learning context. The findings of this study relate to the UK context, in which medical education is usually an undergraduate program; however, exemptions may apply regionally. Furthermore, postgraduate courses or continuous professional development courses and diplomas that specialise in telehealth more broadly often aim to develop effective leadership and/or influence policymaking. A UK example is the postgraduate programme “Leading Digital Transformation in Health and Care for Scotland” (83). Nevertheless, in the context of this study, results partially support previous suggestions to differentiate between “universal fundamentals” and “telehealth specifics” (84). Whilst the former constitutes the education on medical principles such as high-quality, person-centered, ethical care, the latter specifically advocates for an adaptation of these principles in remote consultation settings (29,34,38,70,84–86). By adding practical learning experiences in both simulated and clinical learning environments (24,27,87), future practitioners will be able to adequately analyse the benefits and risks of utilising teleconsultation, learn different strategies for communicating effectively and safely in remote settings, and explore discipline-specific applications of teleconsultations (73).

Continuous educational research is conducted to further develop evidence-informed education for undergraduate medical students including teleconsultation and the more broader realm of telehealth (27,87,88). Discussion at medical schools and clinical placements should be encouraged including the impact of teleconsultations on patient confidentiality and safety, the accessibility to healthcare, the assessment of the clinical appropriateness of teleconsultations, and the implications on clinical workflows. In any case, the use of teleconsultation should be evaluated thoroughly to not unintendedly increase the digital divide in vulnerable population groups and therefore increase health inequalities (73).

### Study limitations

The findings provide a cross-sectional picture of undergraduate medical students’ and medical educators’ opinions of teleconsultation practice and education within the UK. However, the response rate of the online questionnaire was low. This might be explained by limitations of the sampling strategy, survey fatigue, or the stress on educators and students involved in the healthcare and education system during the peak of the COVID-19 pandemic. Additionally, the participants were unevenly spread across the four UK countries which impeded a justification for sub-group analyses. The snowball sampling techniques and self-selection applied to the semi-structured interviews could have limited data saturation as important stakeholders and participants may have withheld their opinions or may not have been reached by the online questionnaire in the first place. Furthermore, the period for data collection extended over 13 months and as the pandemic unfolded, changes in opinions might have occurred. However, an analysis of changes over time was not the aim of this study.

## Conclusion

With the ongoing development of a digitised health and social care system, there is an increasing demand for a skilled workforce. The medical educators and medical students in this mixed-methods study perceived teleconsultation ambiguously, highlighting the advantages but also the fundamental disadvantages of teleconsultations in practice This ambiguity may be reflective of when the study took place during a time when teleconsultations were a necessity during COVID-19 restrictions. Teleconsultation should not be a replacement for in-person health and social care where the latter is needed, nor should teleconsultation education replace the teaching of in-person consultation skills. However, medical education should ensure that the future workforce is adequately prepared for effective and safe teleconsultation practice and also ensure that students are equipped with the knowledge to make choices around the appropriate use of remote consultations over in-person consultations.

## Data Availability

Research data underpinning the PhD thesis are available from 9 May 2028 at [https://doi.org/10.17630/84eb74f3-e316-4618-a112-19b2f24377ac](https://doi.org/10.17630/84eb74f3-e316-4618-a112-19b2f24377ac)

[https://doi.org/10.17630/84eb74f3-e316-4618-a112-19b2f24377ac](https://doi.org/10.17630/84eb74f3-e316-4618-a112-19b2f24377ac) 

*   Received March 28, 2024.
*   Revision received March 28, 2024.
*   Accepted April 1, 2024.


*   © 2024, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)

## References

1.  1.Bradford N, Caffery L, Smith A. Telehealth services in rural and remote Australia: A systematic review of models of care and factors influencing success and sustainability. Rural Remote Health. 2016;16:3808–31.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.22605/RRH3808&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

2.  2.Glinkowski W, Pawlowska K, Kozlowska L. Telehealth and telenursing perception and knowledge among university students of nursing in Poland. Telemed E-Health. 2013 Jul;19(7):523–9.
    
    

3.  3.Rutledge C, Kott K, Schweickert P, Poston R, Fowler C, Haney T. Telehealth and eHealth in nurse practitioner training: current perspectives. Adv Med Educ Pract. 2017 Apr;2017(8):399–409.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

4.  4.The Scottish Government. A national telehealth and telecare delivery plan for Scotland to 2015: Driving improvement, integration and innovation [Internet]. Edinburgh; 2012 Dec [cited 2024 Jan 31]. Available from: [https://www.gov.scot/publications/national-telehealth-telecare-delivery-plan-scotland-2016-driving-improvement-integration-innovation/](https://www.gov.scot/publications/national-telehealth-telecare-delivery-plan-scotland-2016-driving-improvement-integration-innovation/)
    
    

5.  5.World Health Organization and International Telecommunication Union. WHO-ITU global standard for accessibility of telehealth services [Internet]. Geneva; 2022 Jan [cited 2024 Jan 28]. Available from: [https://www.who.int/publications/i/item/9789240050464](https://www.who.int/publications/i/item/9789240050464)
    
    

6.  6.Honeyman M, Dunn P, McKenna H. A digital NHS? An introduction to the digital agenda and plans for implementation [Internet]. 2016 [cited 2024 Mar 10]. Available from: [https://www.kingsfund.org.uk/insight-and-analysis/reports/digital-nhs](https://www.kingsfund.org.uk/insight-and-analysis/reports/digital-nhs)
    
    

7.  7.Imison C, Castle-Clarke S, Watson R, Edwards N. Delivering the benefits of digital health care [Internet]. 2016 Feb [cited 2024 Feb 28]. Available from: [https://www.nuffieldtrust.org.uk/research/delivering-the-benefits-of-digital-health-care](https://www.nuffieldtrust.org.uk/research/delivering-the-benefits-of-digital-health-care)
    
    

8.  8.Karan A, Wadhera RK. Healthcare system stress due to Covid-19: Evading an evolving crisis. J Hosp Med. 2021 Feb;16(2):127–127.
    
    

9.  9.Contreras CM, Metzger GA, Beane JD, Dedhia PH, Ejaz A, Pawlik TM. Telemedicine: Patient-provider clinical engagement during the COVID-19 pandemic and beyond. J Gastrointest Surg. 2020 May;24(7):1692–7.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11605-020-04623-5&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

10. 10.Fisk M, Livingstone A, Pit SW. Telehealth in the context of COVID-19: changing perspectives in Australia, the United Kingdom, and the United States. J Med Internet Res. 2020 Jun;22(6):e19264.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/19264&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

11. 11.Greenhalgh T, Wherton J, Shaw S, Morrison C. Video consultations for COVID-19. BMJ. 2020 Mar;368:m998.
    
    

12. 12.NHS England. GP Patient Survey [Internet]. 2022 [cited 2024 Jan 31]. Available from: [https://gp-patient.co.uk/surveysandreports2022](https://gp-patient.co.uk/surveysandreports2022)
    
    

13. 13.Phuong J, Ordóñez P, Cao J, Moukheiber M, Moukheiber L, Caspi A, et al. Telehealth and digital health innovations: A mixed landscape of access. Kalua K, editor. PLOS Digit Health. 2023 Dec 15;2(12):e0000401.
    
    

14. 14.World Health Organization. Recommendations on digital interventions for health system strengthening [Internet]. 2019 [cited 2024 Mar 10]. Available from: [https://www.who.int/publications/i/item/9789241550505](https://www.who.int/publications/i/item/9789241550505)
    
    

15. 15.Lurie N, Carr BG. The role of telehealth in the medical response to disasters. JAMA Intern Med. 2018 Jun;178(6):745–6.
    
    

16. 16.Terry DL, Buntoro SP. Perceived usefulness of telehealth among rural medical providers: barriers to use and associations with provider confidence. J Technol Behav Sci. 2021 Jun;6(4):567–71.
    
    

17. 17.Deldar K, Bahaadinbeigy K, Tara SM. Teleconsultation and clinical decision making: A systematic review. Acta Inform Medica. 2016 Jul;24(4):286–92.
    
    

18. 18. Carrillo De Albornoz S, Sia KL, Harris A. The effectiveness of teleconsultations in primary care: Systematic review. Fam Pract. 2022 Jan;39(1):168–82.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/fampra/cmab077&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

19. 19.Kludacz-Alessandri M, Hawrysz L, Korneta P, Gierszewska G, Pomaranik W, Walczak R. The impact of medical teleconsultations on general practitioner-patient communication during COVID-19: A case study from Poland. PLOS ONE. 2021 Jul;16(7):e0254960.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0254960&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

20. 20.Farr M, Banks J, Edwards HB, Northstone K, Bernard E, Salisbury C, et al. Implementing online consultations in primary care: A mixed-method evaluation extending normalisation process theory through service co-production. BMJ Open. 2018 Mar;8(3):e019966.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOC8zL2UwMTk5NjYiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wNC8wMS8yMDI0LjAzLjI4LjI0MzA1MDU2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

21. 21.Marshall M, Shah R, Stokes-Lampard H. Online consulting in general practice: Making the move from disruptive innovation to mainstream service. BMJ. 2018 Mar;k1195.
    
    

22. 22.Wherton J, Greenhalgh T. Evaluation of the Attend Anywhere/Near Me video consulting service in Scotland, 2019-20 [Internet]. Edinburgh: The Scottish Government; 2020 Jul [cited 2024 Jan 15]. Available from: [https://www.gov.scot/publications/evaluation-attend-anywhere-near-video-consulting-service-scotland-2019-20-main-report/](https://www.gov.scot/publications/evaluation-attend-anywhere-near-video-consulting-service-scotland-2019-20-main-report/)
    
    

23. 23.The Scottish Government. Urgent care services - Near Me in NHS Greater Glasgow and Clyde’s flow navigation centre: good practice example [Internet]. Edinburgh; 2022 Jul [cited 2024 Jan 31]. Available from: [https://www.gov.scot/publications/near-redesign-urgent-care-integrating-everyday-use-nhs-greater-glasgow-clydes-flow-navigation-centre-good-practice-example/pages/3/](https://www.gov.scot/publications/near-redesign-urgent-care-integrating-everyday-use-nhs-greater-glasgow-clydes-flow-navigation-centre-good-practice-example/pages/3/)
    
    

24. 24.Gunner CK, Eisner E, Watson AJ, Duncan JL. Teaching webside manner: Development and initial evaluation of a video consultation skills training module for undergraduate medical students. Med Educ Online. 2021 Jul;26(1):1954492.
    
    

25. 25.Lee IR, Kim HW, Lee Y, Koyanagi A, Jacob L, An S, et al. Changes in undergraduate medical education due to COVID-19: A systematic review. Eur Rev Med Pharmacol Sci. 2021 Jun;25(12):4426–34.
    
    

26. 26.Rutledge C, Hawkins EJ, Bordelon M, Gustin TS. Telehealth education: An interprofessional online immersion experience in response to COVID-19. J Nurs Educ. 2020 Sep;59(10):570– 6.
    
    

27. 27.Wetzlmair LC, O’Carroll V, O’Malley AS, Murray S. Teleconsultation in health and social care professions education: A systematic review. Clin Teach. 2022 Jul;19(5):e13519.
    
    

28. 28.DeJong C, Lucey CR, Adams Dudley R. Incorporating a new technology while doing no harm, virtually. JAMA. 2015 Dec;314(22):2351–2.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2015.13572&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26647252&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

29. 29.Davies A, Mueller J, Moulton G. Core competencies for clinical informaticians: A systematic review. Int J Med Inf. 2020 Sep;141:104237.
    
    

30. 30.NHS Education for Scotland. Workforce Plan 2019/20 - NHS Education for Scotland Strategy 2019-2024 [Internet]. Edinburgh; 2019 Aug [cited 2024 Jan 20]. Available from: [https://www.nes.scot.nhs.uk/media/gxoj24tm/nes-workforce-plan-2019\_20.pdf](https://www.nes.scot.nhs.uk/media/gxoj24tm/nes-workforce-plan-2019_20.pdf)
    
    

31. 31.Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: A systematic review of systematic reviews (an update. Implement Sci. 2016 Oct;11(1):146–58.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

32. 32.Terry C, Cain J. The emerging issue of digital empathy. Am J Pharm Educ. 2016 May;80(4):58–62.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpwZSI7czo1OiJyZXNpZCI7czo3OiI4MC80LzU4IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDQvMDEvMjAyNC4wMy4yOC4yNDMwNTA1Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

33. 33.Thomas EE, Haydon HM, Mehrotra A, Caffery LJ, Snoswell CL, Banbury A, et al. Building on the momentum: Sustaining telehealth beyond COVID-19. J Telemed Telecare. 2022 May;28(4):301–8.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

34. 34.Brunner M, McGregor D, Keep M, Janssen A, Spallek H, Quinn D, et al. An eHealth capabilities framework for graduates and health professionals: Mixed-methods study. J Med Internet Res. 2018 May;20(5):e10229.
    
    

35. 35.Galpin K, Sikka N, King SL, Horvath KA, Shipman SA, Committee ATA. Expert consensus: Telehealth skills for health care professionals. Telemed J E Health. 2021 Jul;27(7):820–4.
    
    

36. 36.Noronha C, Lo MC, Nikiforova T, Jones D, Nandiwada DR, Leung TI, et al. Telehealth competencies in medical education: New frontiers in faculty development and learner assessments. J Gen Intern Med. 2022 Apr;37(12):3168–73.
    
    

37. 37.Poncette AS, Glauert DL, Mosch L, Braune K, Balzer F, Back D. Undergraduate medical competencies in digital health and curricular module development: Mixed methods study. Jounal Med Internet Res. 2020 Oct;22(10):e22161.
    
    

38. 38.Pit SW, Velovski S, Cockrell K, Bailey J. A qualitative exploration of medical students’ placement experiences with telehealth during COVID-19 and recommendations to prepare our future medical workforce. BMC Med Educ. 2021 Aug;21(1):431–44.
    
    

39. 39.Muntz MD, Franco J, Ferguson CC, Ark TK, Kalet A. Telehealth and medical student education in the time of COVID-19 - and beyond. Acad Med. 2021 Dec;96(12):1655–9.
    
    

40. 40.Chike-Harris KE, Durham C, Logan A, Smith G, DuBose-Morris R. Integration of telehealth education into the health care provider curriculum: A review. Telemed E-Health. 2021 Feb;27(2):137–49.
    
    

41. 41.Levy S. Nurse Education in the Digital Age—A Perspective From the United Kingdom. In: Shachak A, Borycki EM, Reis SP, editors. Health Professionals’ Education in the Age of Clinical Information Systems, Mobile Computing and Social Networks [Internet]. Elsevier; 2017 [cited 2024 Mar 1]. p. 289–97. Available from: [https://linkinghub.elsevier.com/retrieve/pii/B9780128053621000140](https://linkinghub.elsevier.com/retrieve/pii/B9780128053621000140)
    
    

42. 42.Almathami HKY, Win KT, Vlahu-Gjorgievska E. Barriers and facilitators that influence telemedicine-based, real-time, online consultation at patients’ homes: Systematic literature review. J Med Internet Res. 2020 Feb;22(2):e16407.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/16407&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

43. 43.Johnson RB, Onwuegbuzie AJ. Mixed methods research: A research paradigm whose time has come. Educ Res. 2004 Oct;33(7):14–26.
    
    

44. 44.Johnson RB, Onwuegbuzie AJ, Turner LA. Toward a definition of mixed methods research. J Mix Methods Res. 2007 Apr;1(2):112–33.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1558689806298224&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000208042200002&link_type=ISI) 

45. 45.Creswell JD, Creswell JW. Research design: Qualitative, quantitative, and mixed methods approaches. Sixth edition. Sage Publications; 2023.
    
    

46. 46.Morgan DL. Integrating qualitative and quantitative methods. A pragmatic approach. SAGE Publications, Inc; 2014.
    
    

47. 47.Alwashmi MF, Hawboldt J, Davis E, Fetters MD. The iterative convergent design for mobile health usability testing: Mixed methods approach. JMIR MHealth UHealth. 2019 Apr;7(4):e11656.
    
    

48. 48.Proudfoot K. Inductive/deductive hybrid thematic analysis in mixed methods research. J Mix Methods Res. 2022 Sep;17(3):308–26.
    
    

49. 49.Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13(3):319–40.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2307/249008&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1989CC00400006&link_type=ISI) 

50. 50.Worthington RL, Whittaker TA. Scale development research: A content analysis and recommendations for best practices. Couns Psychol. 2006 Nov;34(6):806–38.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0011000006288127&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000241560500002&link_type=ISI) 

51. 51.Qualtrics. Qualtrics [Internet]. 2020 [cited 2023 Jul 1]. Available from: [https://www.qualtrics.com/uk/](https://www.qualtrics.com/uk/)
    
    

52. 52.Microsoft. Microsoft Teams [Internet]. 2022 [cited 2023 Jul 1]. Available from: [https://www.microsoft.com/en/microsoft-teams/log-in](https://www.microsoft.com/en/microsoft-teams/log-in)
    
    

53. 53.Guest G, MacQueen KM, Namey EE. Applied Thematic Analysis. SAGE Publications, Inc; 2012.
    
    

54. 54.Taherdoost H. Sampling methods in research methodology; How to choose a sampling technique for research. Int J Acad Res Manag. 2016 Apr;5(2):18–27.
    
    

55. 55.Boone HN, Boone DA. Alayzing likert scale. J Ext. 2012 Apr;50(2).
    
    

56. 56.Khamis H. Measures of association: How to choose? J Diagn Med Sonogr. 2008 Mar;24(3):155–62.
    
    

57. 57.Clark-Carter D. Measures of Central Tendency. In: Peterson P, Baker E, McGaw B, editors. International Encyclopedia of Education. Third Edition. Elsevier; 2010. p. 264–6.
    
    

58. 58.Corp IBM. IBM SPSS Statistics for Windows [Internet]. IBM; 2019 [cited 2023 Jul 1]. Available from: [https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-26](https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-26)
    
    

59. 59.Microsoft. Microsoft Excel [Internet]. 2021 [cited 2023 Jul 1]. Available from: [https://office.microsoft.com/excel](https://office.microsoft.com/excel)
    
    

60. 60.Fereday J, Muir-Cochrane E. Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods. 2006 Mar;5(1):80–92.
    
    

61. 61.Lumivero. NVivo [Internet]. 2023 [cited 2023 Nov 7]. Available from: [https://lumivero.com/products/nvivo/](https://lumivero.com/products/nvivo/)
    
    

62. 62.Kissi J, Dai B, Dogbe CS, Banahene J, Ernest O. Predictive factors of physicians’ satisfaction with telemedicine services acceptance. Health Inform J. 2019 Dec;26(3):1866– 80.
    
    

63. 63.Bostan S, Yesildag AY, Balci F. Family physicians’ perspectives on the pros, cons, and application areas of telemedicine: A qualitative study [Internet]. 2024 [cited 2024 Mar 13]. Available from: [https://www.liebertpub.com/doi/10.1089/tmj.2023.0574](https://www.liebertpub.com/doi/10.1089/tmj.2023.0574)
    
    

64. 64.Judijanto L, Anurogo D, Zani BN, Muhammad D, Ningsih KP. Implementation of telemedicine in health services: Challenges and opportunities. J World Future Med Health Nurs. 2023 Dec;2(1):37–50.
    
    

65. 65.Du Y, Gu Y. The development of evaluation scale of the patient satisfaction with telemedicine: A systematic review. BMC Med Inform Decis Mak. 2024 Feb;24(1):31.
    
    

66. 66.General Medical Council. Outcomes for graduates 2018 [Internet]. 2020 [cited 2024 Mar 1]. Available from: [https://www.gmc-uk.org/education/standards-guidance-and-curricula/standards-and-outcomes/outcomes-for-graduates](https://www.gmc-uk.org/education/standards-guidance-and-curricula/standards-and-outcomes/outcomes-for-graduates)
    
    

67. 67.Barney A, Buckelew S, Mesheriakova V, Raymond-Flesch M. The COVID-19 pandemic and rapid implementation of adolescent and young adult telemedicine: Challenges and opportunities for innovation. J Adolesc Health. 2020 Aug;67(2):164–71.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jadohealth.2020.05.006&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

68. 68.Penny RA, Bradford NK, Langbecker D. Registered nurse and midwife experiences of using videoconferencing in practice: A systematic review of qualitative studies. J Clin Nurs. 2017 Nov;27(5–6):739–52.
    
    

69. 69.Reilly K, Ebersole A. Confidentiality and privacy considerations for adolescents receiving contraceptive health services via telemedicine: A narrative review. Womens Health [Internet]. 2024 Feb [cited 2024 Mar 13];20. Available from: [http://journals.sagepub.com/doi/10.1177/17455057241233126](http://journals.sagepub.com/doi/10.1177/17455057241233126)
    
    

70. 70.Cummins MR, Soni H, Ivanova J, Ong T, Barrera J, Wilczewski H, et al. Narrative review of telemedicine applications in decentralized research. J Clin Transl Sci. 2024 Jan;8(1):e30– 40.
    
    

71. 71.Kruse CS, Karem P, Shifflett K, Vegi L, Ravi K, Brooks M. Evaluating barriers to adopting telemedicine worldwide: A systematic review. J Telemed Telecare. 2016 Oct;24(1):4–12.
    
    

72. 72.Moffatt JJ, Eley DS. Barriers to the up-take of telemedicine in Australia – A view from providers. Rural Remote Health. 2011 Feb;11(1):1581–7.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21385004&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

73. 73.Gallegos-Rejas VM, Thomas EE, Kelly JT, Smith AC. A multi-stakeholder approach is needed to reduce the digital divide and encourage equitable access to telehealth. J Telemed Telecare. 2023 Jan;29(1):73–8.
    
    

74. 74.Reed ME, Huang J, Graetz I, Lee C, Muelly E, Kennedy C, et al. Patient characteristics associated with choosing a telemedicine visit vs office visit with the same primary care clinicians. JAMA Netw Open. 2020 Jun;3(6):e205873–e205873.
    
    

75. 75.Norman CD, Skinner HA. eHealth literacy: Essential skills for consumer health in a networked world. J Med Internet Res. 2006 Jun;8(2):e9.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/jmir.8.2.e9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16867972&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

76. 76.Haupeltshofer A, Egerer V, Seeling S. Promoting health literacy: What potential does nursing informatics offer to support older adults in the use of technology? A scoping review. Health Inform J. 2020 Jul;26(4):2707–21.
    
    

77. 77.The Scottish Council for Voluntary Organisations. Connecting Scotland - supporting everyone in Scotland to get online. 2024 [cited 2024 Feb 29]. Connecting Scotland - supporting everyone in Scotland to get online. Available from: [https://connecting.scot/](https://connecting.scot/)
    
    

78. 78.Dahlgren G, Whitehead M. The Dahlgren-Whitehead model of health determinants: 30 years on and still chasing rainbows. Public Health. 2021 Oct;199:20–4.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.puhe.2021.08.009&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34534885&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

79. 79.Chu M, Dalwadi S, Profit R, Searle B, Williams H. How should medical education support increasing telemedicine use following COVID-19? An Asian perspective focused on teleconsultation. Int J Digit Health. 2022 Jan;2(1):1–6.
    
    

80. 80.Cowan KE, McKean AJ, Gentry MT, Hilty DM. Barriers to use of telepsychiatry: Clinicians as gatekeepers. Mayo Clin Proc. 2019 Dec;94(12):2510–23.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.mayocp.2019.04.018&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom) 

81. 81.Van Galen LS, Wang CJ, Nanayakkara PWB, Paranjape K, Kramer MHH, Car J. Telehealth requires expansion of physicians’ communication competencies training. Med Teach. 2019 Jun;41(6):714–5.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/0142159X.2018.1481284&link_type=DOI) 

82. 82.Singh M, Collins L, Farrington R, Jones M, Thampy H, Watson P, et al. From principles to practice: Embedding clinical reasoning as a longitudinal curriculum theme in a medical school programme. Diagn Berl. 2021 Jul;9(2):184–94.
    
    

83. 83.NHS Education for Scotland. Postgraduate Programme: Leading Digital Transformation in Health and Care for Scotland (PGCert, PGDip, MSc) [Internet]. 2022 [cited 2024 Mar 12]. Available from: [https://www.nes.scot.nhs.uk/news/postgraduate-programme-leading-digital-transformation-in-health-and-care-for-scotland-pgcert-pgdip-msc/](https://www.nes.scot.nhs.uk/news/postgraduate-programme-leading-digital-transformation-in-health-and-care-for-scotland-pgcert-pgdip-msc/)
    
    

84. 84.Pourmand A, Ghassemi M, Sumon K, Amini SB, Hood C, Sikka N. Lack of telemedicine training in academic medicine: Are we preparing the next generation? Telemed E-Health. 2021 Jan;27(1):62–7.
    
    

85. 85.Davies A, Mueller J, Hassey A, Moulton G. Development of a core competency framework for clinical informatics. BMJ Health Care Inform. 2021 Jul;28(1):e100356.
    
    

86. 86.Moulton G, Hassey A, Davies A. Faculty of Clinical Informatics Core Competency Framework [Internet]. The University of Manchester & Faculty of Clinical Informatics; 2020 [cited 2024 Feb 29]. Available from: [https://zenodo.org/records/3957992](https://zenodo.org/records/3957992)
    
    

87. 87.Bouamra B, Chakroun K,  Medeiros De Bustos E, Dobson J, Rouge JA, Moulin T. Simulation-based teaching of telemedicine for future users of teleconsultation and tele-expertise: Feasibility study. JMIR Med Educ. 2021 Dec;7(4):e30440.
    
    

88. 88. Van Der Kleij RMJJ, Kasteleyn MJ, Meijer E, Bonten TN, Houwink EJF, Teichert M, et al. eHealth in primary care. Part1: Concepts, conditions and challenges. Eur J Gen Pract. 2019 Oct;25(4):179–89.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F01%2F2024.03.28.24305056.atom)