Are remote mental healthcare interventions cost-effective? A systematic review of economic evaluations of remote mental healthcare ================================================================================================================================== * Amy Clark * Rebecca Appleton * Erika Kalocsanyiova * Evdoxia Gkaintatzi * Paul McCrone ## Abstract **Background** Remote interventions known as telemental health care increased in use due to the COVID-19 pandemic when social distancing requirements were in place. Whilst there is some evidence regarding the cost-effectiveness of telemental health prior to the pandemic, there is a need for further evaluation due to the increase in remote care. **Aims** To systematically review the literature to explore whether remote mental health care interventions are cost-effective in terms of incremental cost per quality adjusted life year and in relation to condition specific outcomes compared to usual care or an alternative intervention. **Method** A multilayer search strategy was conducted to build on the searches of a previous systematic review, as well as including grey literature and economic models. Six databases (PubMed, EMBASE, Cochrane Central, PsychINFO, CINAHL, and EconLit) were searched for literature relating to the cost effectiveness of telemental health. Quality appraisal was conducted for all included studies, and findings were synthesised using narrative synthesis. **Results** 7386 studies were identified of which 59 met our inclusion criteria and were included in the synthesis of findings. 45 studies were rated as very good or excellent quality. Of the 59 included studies, 40 indicated that the telemental health intervention was cost-effective, whilst a further 16 suggested the intervention had potential to be cost-effective, but there was some uncertainty in the findings. Three studies reported that the intervention was not cost-effective. **Conclusions** This evidence will be used to inform practice in the UK as we respond to and recover from the COVID-19 pandemic. Keywords * telemental health * QALYs * cost-effectiveness analysis ## Introduction Remote interventions in mental health care, or “tele-mental health” have been implemented for many years and evaluations often include assessment of cost-effectiveness. Telemental health care is defined as any interventions or modes of working where remote technology (e.g. telephone/video call/instant messaging) is used to facilitate direct communication between staff and service users, between service users and peers, or between mental health professionals in different locations. Telemental health has the potential to result in benefits for both service users and clinicians. For example, service users have identified increased flexibility as a key advantage of remote care (1), and a recent rapid realist review (2) found that telemental health care can improve access to mental health support for people who may struggle to travel to face-to-face appointments due to disability, anxiety about travel, or caring or work responsibilities. Reported benefits for staff included an opportunity for more flexible working, less time spent travelling, and communicating with other clinicians at different sites (2, 3). However, it is important to note that telemental health is not suitable for everyone; not all service users may wish for care to be remote and inequalities and digital exclusion should be taken into consideration during the decision to offer mental health care remotely (1, 2). Clinicians have also reported finding telemental health less suitable than face-to-face care when treating trauma, or for service users in crisis or those with psychotic symptoms or severe anxiety (3). The uptake of telemental health care increased rapidly during the COVID-19 pandemic as services moved from largely face-to-face models of care due to social distancing requirements (4, 5). However, a systematic review conducted during the early stages of the pandemic identified a lack of evidence on the cost-effectiveness of telemental health interventions (4). Pre-pandemic evidence regarding the cost-effectiveness of telemental health was also inconclusive. An umbrella review found that in studies conducted prior to the COVID-19 pandemic, evidence regarding the cost-effectiveness of telemental health was mixed (6). Some studies identified lower costs of telemental health care due to savings on travel time for service users, or not needing to take time off work, whereas higher costs in others were attributed to the expense of videoconferencing equipment (6). A further systematic review by Naslund et al (7) identified 26 economic evaluations of telepsychiatry programmes, also prior to the COVID-19 pandemic. They found that 60% reported telepsychiatry programmes to be less expensive and 32% reported they were more expensive than usual care, again primarily due to costs of technology and equipment. This review was limited to peer-reviewed studies that used primary data collection and therefore economic models were excluded. This proposed review therefore aims to build on Naslund et al (7) and bring together further evidence on economic evaluations of remote mental healthcare. We originally set out to answer the following secondary review questions: (i) Does cost-effectiveness of remote mental health care interventions differ by subgroup? (ii) Does cost-effectiveness of remote mental health care interventions differ by model type? (iii) What is the impact of remote technology for staff communication on costs for the health and social care system? (iv) What is the impact of remote technology for mental healthcare interventions on costs for the health and social care system? (v) How is the amount of contact time between service users and mental health care professionals affected by remote interventions? However, given the data reported in the selected studies only research questions ii and iv will be addressed in this paper. ## Methods The review was registered on International prospective register of Systematic Reviews (PROSPERO) ID: CRD42020216755. We developed a multilayer search strategy to capture both new evidence published since the completion of the Naslund et al. (7) searches on 16 March 2018 and to cover additional sources of information excluded by these authors, namely grey literature and simulation models. We used the same search criteria as outlined in Naslund et al. (7) to identify any relevant peer reviewed literature published since 16 March 2018. The following six databases were searched on 8-10 December 2020: PubMed (MEDLINE), EMBASE, Cochrane Central, PsychINFO (EBSCOhost), CINAHL, and EconLit1. An example of the search terms used, as per Naslund et al. (7), is reported in Table S12 of the Supplementary Material. To identify economic models, which were excluded by Naslund et al. (7), the same strategy was used but with a different date range (January 2000 – October 2020) and two additional search terms added to Table S1: “model OR simulation”. Grey literature databases were searched from 1 January 2000 to 31 October 2020 for terms related to “tele” “mental health” and “costs” as outlined in Table S2. MedNAR and Google were also searched using the following terms to improve relevance of results: “cost effective” AND (tele* OR telehealth OR telepsychiatry OR remote OR technology OR digital) AND mental. Records identified in the search were uploaded to Rayyan QCRI, a web-based reference manager system for collaborative systematic review, for de-duplication and blinded screening and study selection. ### Study selection criteria After de-duplication, the results were pre-screened by three student research assistants (MT, SS, and JP). A 25 per cent sample was selected at random and checked for accuracy by the second author (EK). The titles and abstracts of the remaining studies were screened in duplicate by three reviewers (AC, EK and JB) against the eligibility criteria shown in Table 1. Disagreements and records marked as undecided were resolved by discussion or a consensus involving a third author. View this table: [Table 1:](http://medrxiv.org/content/early/2022/12/02/2022.12.01.22282817/T1) Table 1: Eligibility criteria. In the process of abstract and title screening, we identified 21 peer-reviewed cost effectiveness studies published prior to March 2018 which appeared eligible according to the study selection criteria outlined in Naslund et al. (7) but were not included in their review. Given that these studies were identified in grey literature databases not considered by Naslund et al. (7), we decided to assess them for inclusion along with the other studies retained for full text screening. Where full text was not available, the corresponding authors were contacted. If an answer was not obtained following a reminder and a 3-month waiting period, the study was excluded. Reference list searching was undertaken by the second author (EK). The search and study selection process has been documented in Figure 1 using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses PRISMA guidelines (8). ![Figure 1:](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2022/12/02/2022.12.01.22282817/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2022/12/02/2022.12.01.22282817/F1) Figure 1: PRISMA flow chart. ### Data extraction and analysis Partial data extraction was undertaken by three student research assistants (SS, HA and FB) and completed by two of the authors (EG and AC). For 25 per cent of studies a third team member extracted the data in duplicate. Following data extraction, the studies were sorted by intervention type, categorised into study design type (i.e., model/cost utility analysis or cost-effectiveness analysis), and we considered whether the evidence favoured the intervention or not. Given the heterogeneity of the retained studies meta-analysis was not undertaken. As with most reviews of economic evaluations, the results are presented in the form of a narrative synthesis (9). ### Quality appraisal We used the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) (10) checklist to assess different aspects of the retained studies’ quality and reporting. The checklist contains 24 items and accompanying recommendations (see Table S3). The overall quality of evidence was assessed using GRADE certainty ratings (11). Final quality was rated high, moderate, low, or very low. The quality assessment was completed by EK. A 25 per cent sample was selected at random and assessed in duplicate by AC. Where results differed, consensus was reached through discussion or the involvement of a third assessor (PM). ## Results The initial search identified 7386 papers. Duplicates were then removed and after screening by three research assistants (MT, SS, and JP) a further 2462 records were excluded. The random sample of 25% used for checking revealed no conflicts. After the duplicate screening of remaining papers, disagreements (n=26) and records marked as undecided (n=77) were resolved. A total of 59 unique studies met the inclusion criteria (Figure 1). Eight full text papers could not be obtained and reference lists revealed no extra studies. Search 1 identified four grey literature reports, search 2 identified 41 unique studies from the updated Naslund et al (7) search, and search 3 found a further 14 studies of economic models. The non-modelling studies enrolled 5-1514 participants. The USA was the setting for most studies (16), followed by Sweden (10) and the UK (8). For more detailed study characteristics see Table 3. View this table: [Table 2:](http://medrxiv.org/content/early/2022/12/02/2022.12.01.22282817/T2) Table 2: Summary of GRADE certainty ratings. View this table: [Table 3:](http://medrxiv.org/content/early/2022/12/02/2022.12.01.22282817/T3) Table 3: Study characteristics. The majority (n=40) of included studies found that the telemental health intervention was cost-effective. A further 16 studies indicated that there was some uncertainty around whether the intervention was more cost-effective than the comparator, or that the intervention had potential to be cost-effective but there were some biases in how the study was conducted. Only three studies found that the intervention was less cost-effective than the comparator. Table 3 shows that the most common interventions were internet-based therapy and videoconferencing. Twenty-one studies evaluated some form of internet-based/delivered therapy (usually cognitive behavioural therapy). QALYs were used in 13 of these studies, DALYs in two studies, productivity losses in one study, and clinically specific measures in the remaining studies. In eight studies the costs for the intervention were lower than the comparator and outcomes were better, indicating a situation of dominance (12–19). In five studies where the intervention had higher costs and produced better outcomes, the ICER was relatively low and indicated cost-effectiveness (20–24). Two studies indicated that therapy was not cost-effective (25, 26). Videoconferencing and video support to clinicians and patients was evaluated in eight studies. Four of these did not use an outcome measure in the economic analysis and all showed cost savings (27–30). Two other studies reported similar outcomes and lower costs (31, 32). One study, evaluating video coaching for carers of children with behavioural problems, found it to cost saving and outcome improving (27). Another study, found videoconferencing to result in costs per improvement in depression of $222 and per improvement in competence of $437 (33). Studies that evaluated other remote working interventions usually demonstrated costs savings, dominance (costs savings and better outcomes), or higher costs and better outcomes that justified the extra expense. The study by Pot-Kolder et al (34) found the cost per QALY of virtual reality-based CBT for people with psychosis and paranoid symptoms to be nearly €50,000. However, this was deemed to be cost-effective based on a threshold of €80,000 per QALY for conditions such as this. The number of studies taking a societal perspective was 27 with 22 taking a healthcare perspective. QALYs were reported in 31 studies and DALYs in four. The reporting quality of each included article was assessed using the 24-item CHEERS checklist: the proportion of studies that met the criteria for each item is shown in Figure 2. 28 studies were of excellent quality, meaning that they addressed satisfactorily at least 90% of applicable checklist items (12, 13, 15–17, 20–22, 24–27, 31, 34–47). 17 studies (14, 19, 23, 48–61) were of very good quality (70-<90%), and further five studies (18, 62–65); had an acceptable quality of reporting (> 60%). The remaining nine studies failed to address or addressed only partially ten or more relevant checklist items. The least reported items were study perspective (item # 6), currency, price date, and conversion (item # 14), the effects of uncertainty (item # 20), and potential conflicts of interest (item # 24). About 40 % of the studies failed to describe fully the characteristics of the base-case population (item # 4), and the sources and methods used to establish clinical effectiveness (item # 11). Similarly, only 80% of the studies reported in sufficient detail the approaches and data sources used to estimate costs (item # 13) and the analytical methods which supported their evaluation (item # 17). Data for subgroup analyses was not available in most cases (item # 21). Finally, approximately half of the studies either did not name its funding source or failed to disclose the funder’s role in the identification, design, conduct, and reporting of the analysis. The detailed performance of each included study for the CHEERS checklist (10) is shown in Table S4 of the Supplement. The overall quality of evidence was assessed using GRADE certainty ratings (11). The assigned ratings shown in Table 2 were based on the quality of the economic evidence. ![Figure 2:](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2022/12/02/2022.12.01.22282817/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2022/12/02/2022.12.01.22282817/F2) Figure 2: Proportion of included studies that complied with applicable items of the Consolidated Health Economic Evaluation Reporting Standard (CHEERS) checklist. Overall, there was an over reliance on single-study based estimates. The certainty of evidence was downgraded due to several factors, including imprecision (effect estimates coming from one study with a small sample), risk of bias (narrow perspective, cost omissions related to comorbidity, healthcare utilisation and other relevant aspects, losses to follow-up, short time horizon and limited sensitivity analysis), indirectness (cost estimates based on secondary sources rather than actual resource use, and high percentage of missing data imputed) and potential sponsor bias. The main reasons for downgrading for each study rated as moderate or below are summarised in Table 2. ## Discussion This systematic review identified 59 studies to build on the findings from Naslund et al (7) regarding the impact of remote technology for mental healthcare interventions in terms of costs and cost-effectiveness. The review by Naslund et al (7) included 26 studies and found good evidence for cost savings associated with telepsychiatry but somewhat limited evidence of cost-effectiveness. We included a broader range of studies and again these suggest that remote working can result in reduced costs but we also identified a reasonable number of full economic evaluations where remote working was either dominant (cost saving and outcome improving) or to result in incremental cost-effectiveness ratios that are below acceptable thresholds. However, as with Naslund et al (7), the studies were markedly heterogeneous with very different designs and perspectives. The greatest number of studies evaluated some form of internet-delivered therapy (usually cognitive behavioural therapy). These tended to be cost effective as were the various forms of videoconferencing (although many of the latter were limited to cost comparisons). The perspective taken in the studies was fairly evenly split between a societal perspective (generally meaning that lost work time was included) and a healthcare perspective. This is likely to reflect both the expected benefits of remote working but also the different ways in which healthcare decisions are made in each country. While the latter consideration is important, from an economic point of view we might expect remote working to save patient time and this may include time in work. As such, a societal perspective seems appropriate in this area. Decision makers often base recommendations on the relationship between costs and QALYs and the studies reviewed here reflect this with 31 using QALYs as the main economic outcome measure. This is interesting especially given some concerns over the use of QALYs in mental health research (66, 67). Most of these concerns though are with the sue of QALYs in studies of interventions for schizophrenia and other severe mental illnesses. As stated above, internet-based cognitive behavioural therapy was most commonly evaluated, and this usually provided for people with depression or anxiety. QALYs do tend to work reasonably well for these conditions. ### Implications Evaluations of remote working interventions for mental health problems have been criticised for being methodologically limited and this may have held back developments in this area (68). It is from this review that the amount of cost-effectiveness research has increased substantially, and the evidence base is getting stronger. However, methodological issues persist. The COVID-19 pandemic has clearly led to remote working being given more prominence. While services will to some extent revert back to usual ways of working, some aspects of care delivery that have emerged since early 2020 will most likely remain. In the UK, the charity Mind has identified challenges including the extent to which people have good access to engage with digital approaches, the quality of care delivered in this way not always being of a high standard, and potential breaches of confidentiality (69). As demands on health services continue to increase, it is likely that innovations such as those reviewed here will be needed more and more and it is thus encouraging that on balance they do appear to represent reasonable value for money. ### Limitations We based our search terms on those used previously by Naslund et al (7). This was appropriate in that we were wanting to update that review but also to include studies from a broader range of sources. However, the strategy may have been too specific and might have missed economic models published prior to 2018. Some terms were excluded that could have been relevant, particularly those relating to the use of social media and platforms such as Zoom or Microsoft Teams. Another limitation is that some interventions/services only included remote working as a component of a wider package of care or support. It was appropriate to include such studies in order to obtain a broad overview of remote working but identifying the specific impact of remote working is challenging in such studies. As with other reviews of economic evaluations, included studies were very heterogeneous. This meant that we could only provide a narrative review rather than a more formal synthesis of findings. ## Conclusion This review has found numerous studies assessing the costs and cost-effectiveness of remote working interventions in mental health. Such approaches appear to be cost-effective although methodological quality of studies needs to be enhanced. ## Data Availability This is a systematic review and no primary data have been colelcted. ## Data Availability Data availability is not applicable to this article as no new data were created or analysed in this study. ## Author Contributions The study was developed by AC and PM. Papers were identified and data extracted by AC, EK, EG, and PM. The manuscript was written by AC, RA, EK, EG and PM. ## Funding This paper presents independent research commissioned and funded by the National Institute for Health Research (NIHR) Policy Research Programme, conducted by the NIHR Policy Research Unit (PRU) in Mental Health. AC was funded through an NIHR research fellowship. The views expressed are those of the authors and not necessarily those of the NIHR, the Department of Health and Social Care or its arm’s length bodies, or other government departments. ## Declaration of interest None ## Supplemental material View this table: [Table S1:](http://medrxiv.org/content/early/2022/12/02/2022.12.01.22282817/T4) Table S1: Naslund et al. (2020) search strategy used in Medline View this table: [Table S2:](http://medrxiv.org/content/early/2022/12/02/2022.12.01.22282817/T5) Table S2: Grey literature search strategy View this table: [Table S3:](http://medrxiv.org/content/early/2022/12/02/2022.12.01.22282817/T6) Table S3: Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist View this table: [Table S4.](http://medrxiv.org/content/early/2022/12/02/2022.12.01.22282817/T7) Table S4. CHEERS checklist: Results ## Acknowledgements Joseph Botham assisted with the initial search and data extraction. UCL post-graduate placement students provided invaluable contribution to this study. Initial title and abstract screening was completed by Magdalena Tomoskova, Jiping Mo and Sypros Spyridonids. Data extraction support was completed by Hana Afra, Spyros Spyridonidis and Fatima Bashir. ## Footnotes * 1 The following databases used in Naslund et al. (2020) were not included in this review due to organisational access and the number of duplicates already identified: Web of Science, Health Economic Evaluations Database, Cost-Effectiveness Analysis Registry, Research Papers in Economics (RcPEc), and European Network of Health Economic Evaluation Database (EURONHEED). * 2 In CINAHL we were unable to apply MeSH terms, and in EconLit the simplified grey literature strategy was used (see Table S2). * Received December 1, 2022. * Revision received December 1, 2022. * Accepted December 2, 2022. * © 2022, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NoDerivs 4.0 International), CC BY-ND 4.0, as described at [http://creativecommons.org/licenses/by-nd/4.0/](http://creativecommons.org/licenses/by-nd/4.0/) ## References 1. 1.Vera San Juan N, Shah P, Schlief M, Appleton R, Nyikavaranda P, Birken M, et al. Service user experiences and views regarding telemental health during the COVID-19 pandemic: A co-produced framework analysis. PloS one. 2021;16(9):e0257270. 2. 2.Schlief M, Saunders KR, Appleton R, Barnett P, San Juan NV, Foye U, et al. What works for whom with telemental health? A rapid realist review. medRxiv. 2022. 3. 3.Feijt M, De Kort Y, Bongers I, Bierbooms J, Westerink J, IJsselsteijn W. Mental health care goes online: Practitioners’ experiences of providing mental health care during the COVID-19 pandemic. Cyberpsychology, Behavior, and Social Networking. 2020;23(12):860–4. 4. 4.Appleton R, Williams J, San Juan NV, Needle JJ, Schlief M, Jordan H, et al. Implementation, adoption, and perceptions of telemental health during the COVID-19 pandemic: systematic review. Journal of medical Internet research. 2021;23(12):e31746. 5. 5.Zangani C, Ostinelli EG, Smith KA, Hong JS, Macdonald O, Reen G, et al. Impact of the COVID-19 Pandemic on the Global Delivery of Mental Health Services and Telemental Health: Systematic Review. JMIR mental health. 2022;9(8):e38600. 6. 6.Barnett P, Goulding L, Casetta C, Jordan H, Sheridan-Rains L, Steare T, et al. Implementation of telemental health services before COVID-19: rapid umbrella review of systematic reviews. Journal of medical Internet research. 2021;23(7):e26492. 7. 7.Naslund JA, Mitchell LM, Joshi U, Nagda D, Lu C. Economic evaluation and costs of telepsychiatry programmes: A systematic review. Journal of telemedicine and telecare. 2022;28(5):311–30. 8. 8.Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic reviews. 2021;10(1):1–11. 9. 9.Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme Version. 2006;1(1):b92. 10. 10.Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS) statement. International journal of technology assessment in health care. 2013;29(2):117–22. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1017/S0266462313000160&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23587340&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 11. 11.Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Bmj. 2008;336(7650):924–6. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMzYvNzY1MC85MjQiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8xMi8wMi8yMDIyLjEyLjAxLjIyMjgyODE3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 12. 12.Lindsäter E, Axelsson E, Salomonsson S, Santoft F, Ljótsson B, Åkerstedt T, et al. Cost-effectiveness of therapist-guided internet-based cognitive behavioral therapy for stress-related disorders: secondary analysis of a randomized controlled trial. Journal of medical Internet research. 2019;21(9):e14675. 13. 13.Baumann M, Stargardt T, Frey S. Cost–utility of internet-based cognitive behavioral therapy in unipolar depression: A Markov model Simulation. Applied health economics and health policy. 2020;18(4):567–78. 14. 14.Mohr DC, Lattie EG, Tomasino KN, Kwasny MJ, Kaiser SM, Gray EL, et al. A randomized noninferiority trial evaluating remotely-delivered stepped care for depression using internet cognitive behavioral therapy (CBT) and telephone CBT. Behaviour Research and Therapy. 2019;123:103485. 15. 15.Jolstedt M, Wahlund T, Lenhard F, Ljótsson B, Mataix-Cols D, Nord M, et al. Efficacy and cost-effectiveness of therapist-guided internet cognitive behavioural therapy for paediatric anxiety disorders: a single-centre, single-blind, randomised controlled trial. The Lancet Child & Adolescent Health. 2018;2(11):792–801. 16. 16.van Luenen S, Kraaij V, Garnefski N, Spinhoven P, van den Akker–van ME. Cost-utility of a guided Internet-based intervention in comparison with attention only for people with HIV and depressive symptoms: A randomized controlled trial. Journal of psychosomatic research. 2019;118:34–40. 17. 17.Lalouni M, Ljótsson B, Bonnert M, Ssegonja R, Benninga M, Bjureberg J, et al. Clinical and cost effectiveness of online cognitive behavioral therapy in children with functional abdominal pain disorders. Clinical Gastroenterology and Hepatology. 2019;17(11):2236–44. e11. 18. 18.Zhou Y-X. Efficacy and cost-effectiveness of internet-based cognitive behavioral therapy for obsessive-compulsive disorder. Journal of Shanghai Jiaotong University (Medical Science). 2019:622–8. 19. 19.Kumar S, Jones Bell M, Juusola JL. Mobile and traditional cognitive behavioral therapy programs for generalized anxiety disorder: A cost-effectiveness analysis. PloS one. 2018;13(1):e0190554. 20. 20.Hedman E, Andersson E, Ljótsson B, Axelsson E, Lekander M. Cost effectiveness of internet-based cognitive behaviour therapy and behavioural stress management for severe health anxiety. BMJ open. 2016;6(4):e009327. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiNi80L2UwMDkzMjciO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8xMi8wMi8yMDIyLjEyLjAxLjIyMjgyODE3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 21. 21.Nobis S, Ebert DD, Lehr D, Smit F, Buntrock C, Berking M, et al. Web-based intervention for depressive symptoms in adults with types 1 and 2 diabetes mellitus: a health economic evaluation. The British Journal of Psychiatry. 2018;212(4):199–206. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 22. 22.Kraepelien M, Mattsson S, Hedman-Lagerlöf E, Petersson IF, Forsell Y, Lindefors N, et al. Cost-effectiveness of internet-based cognitive–behavioural therapy and physical exercise for depression. BJPsych Open. 2018;4(4):265–73. 23. 23.Mihalopoulos C, Shih STF, Kiropoulos L, Blashki G, Meadows G, Gunn J. Exploratory economic analyses of two primary care mental health projects: implications for sustainability. Medical Journal of Australia. 2005;183:S73–S6. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16296957&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000234373800007&link_type=ISI) 24. 24.Richards D, Bower P, Chew-Graham C, Gask L, Lovell K, Cape J, et al. Clinical effectiveness and cost-effectiveness of collaborative care for depression in UK primary care (CADET): a cluster randomised controlled trial. Health Technology Assessment. 2016;20(14):1–192. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3310/hta20330&link_type=DOI) 25. 25.Kolovos S, van Dongen JM, Riper H, Buntrock C, Cuijpers P, Ebert DD, et al. Cost effectiveness of guided Internet-based interventions for depression in comparison with control conditions: An individual–participant data meta-analysis. Depression and anxiety. 2018;35(3):209–19. 26. 26.Axelsson E, Andersson E, Ljótsson B, Hedman-Lagerlöf E. Cost-effectiveness and long-term follow-up of three forms of minimal-contact cognitive behaviour therapy for severe health anxiety: Results from a randomised controlled trial. Behaviour research and therapy. 2018;107:95–105. 27. 27.Olthuis JV, McGrath PJ, Cunningham CE, Boyle MH, Lingley-Pottie P, Reid GJ, et al. Distance-delivered parent training for childhood disruptive behavior (Strongest Families™): a randomized controlled trial and economic analysis. Journal of abnormal child psychology. 2018;46(8):1613–29. 28. 28.Jardine I, Griffiths J, Kelly J, Navein J. Telemedicine options appraisal for a health district--a strategic clinical and business case. J Telemed Telecare. 2001;7 Suppl 1:16–9. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1258/1357633011937326&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11747647&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 29. 29.Mielonen M-L, Ohinmaa A, Moring J, Isohanni M. Psychiatric inpatient care planning via telemedicine. Journal of Telemedicine and Telecare. 2000;6(3):152–7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1258/1357633001935248&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10912333&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 30. 30.Shore JH, Brooks E, Savin DM, Manson SM, Libby AM. An economic evaluation of telehealth data collection with rural populations. Psychiatr Serv. 2007;58(6):830–5. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1176/appi.ps.58.6.830&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17535944&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 31. 31.Egede LE, Dismuke CE, Walker RJ, Acierno R, Frueh BC. Cost-effectiveness of behavioral activation for depression in older adult veterans: in-person care versus telehealth. The Journal of clinical psychiatry. 2018;79(5):3853. 32. 32.Wilson JA. Psychoeducational dissemination across distance: The viability of telehealth with the deaf population: Brigham Young University; 2008. 33. 33.Shaw CA, Williams KN, Lee RH, Coleman CK. Cost-effectiveness of a telehealth intervention for in-home dementia care support: Findings from the FamTechCare clinical trial. Research in nursing & health. 2021;44(1):60–70. 34. 34.Pot-Kolder R, Veling W, Geraets C, Lokkerbol J, Smit F, Jongeneel A, et al. Cost-effectiveness of virtual reality cognitive behavioral therapy for psychosis: health-economic evaluation within a randomized controlled trial. Journal of medical Internet research. 2020;22(5):e17098. 35. 35.Ontario HQ. Internet-delivered cognitive behavioural therapy for major depression and anxiety disorders: a health technology assessment. Ontario health technology assessment series. 2019;19(6):1. 36. 36.Axelsson E, Andersson E, Ljótsson B, Björkander D, Hedman-Lagerlöf M, Hedman-Lagerlöf E. Effect of internet vs face-to-face cognitive behavior therapy for health anxiety: a randomized noninferiority clinical trial. JAMA psychiatry. 2020;77(9):915–24. 37. 37.Hedman-Lagerlöf M, Hedman-Lagerlöf E, Ljótsson B, Wicksell RK, Flink I, Andersson E. Cost-effectiveness and cost-utility of internet-delivered exposure therapy for fibromyalgia: results from a randomized, controlled trial. The Journal of Pain. 2019;20(1):47–59. 38. 38.Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, et al. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health Technology Assessment (Winchester, England). 2006;10(33):iii, xi-xiv, 1. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3310/hta10340&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16595081&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000238229500001&link_type=ISI) 39. 39.Klein NS, Bockting CL, Wijnen B, Kok GD, van Valen E, Riper H, et al. Economic evaluation of an internet-based preventive cognitive therapy with minimal therapist support for recurrent depression: randomized controlled trial. Journal of medical Internet research. 2018;20(11):e10437. 40. 40.Kooistra LC, Wiersma JE, Ruwaard J, Neijenhuijs K, Lokkerbol J, van Oppen P, et al. Cost and effectiveness of blended versus standard cognitive behavioral therapy for outpatients with depression in routine specialized mental health care: pilot randomized controlled trial. Journal of medical Internet research. 2019;21(10):e14261. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 41. 41.Lee Y-C, Gao L, Dear BF, Titov N, Mihalopoulos C. The cost-effectiveness of the online MindSpot Clinic for the treatment of depression and anxiety in Australia. J Ment Health Policy Econ. 2017;20(4):155–66. 42. 42.Morriss R, Patel S, Malins S, Guo B, Higton F, James M, et al. Clinical and economic outcomes of remotely delivered cognitive behaviour therapy versus treatment as usual for repeat unscheduled care users with severe health anxiety: a multicentre randomised controlled trial. BMC medicine. 2019;17(1):1–13. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12916-019-1466-7&link_type=DOI) 43. 43.Richards D, Enrique A, Eilert N, Franklin M, Palacios J, Duffy D, et al. A pragmatic randomized waitlist-controlled effectiveness and cost-effectiveness trial of digital interventions for depression and anxiety. NPJ digital medicine. 2020;3(1):1–10. 44. 44.Smit F, Lokkerbol J, Riper H, Majo C, Boon B, Blankers M. Modeling the cost-effectiveness of health care systems for alcohol use disorders: how implementation of eHealth interventions improves cost-effectiveness. Journal of Medical Internet Research. 2011;13(3):e1694. 45. 45.Holst A, Björkelund C, Metsini A, Madsen J-H, Hange D, Petersson E-LL, et al. Cost-effectiveness analysis of internet-mediated cognitive behavioural therapy for depression in the primary care setting: results based on a controlled trial. BMJ open. 2018;8(6):e019716. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOC82L2UwMTk3MTYiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8xMi8wMi8yMDIyLjEyLjAxLjIyMjgyODE3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 46. 46.Kählke F, Buntrock C, Smit F, Berking M, Lehr D, Heber E, et al. Economic evaluation of an internet-based stress management intervention alongside a randomized controlled trial. JMIR mental health. 2019;6(5):e10866. 47. 47.Mukuria C, Brazier J, Barkham M, Connell J, Hardy G, Hutten R, et al. Cost-effectiveness of an improving access to psychological therapies service. The British Journal of Psychiatry. 2013;202(3):220–7. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6ImJqcHJjcHN5Y2giO3M6NToicmVzaWQiO3M6OToiMjAyLzMvMjIwIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMTIvMDIvMjAyMi4xMi4wMS4yMjI4MjgxNy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 48. 48.Dowd WN, Cowell AJ, Regan D, Moran K, Slevin P, Doyle G, et al. An exploratory cost-effectiveness analysis of the connected health intervention to improve care for people with dementia: a simulation analysis. Health Services and Outcomes Research Methodology. 2018;18(1):47–62. 49. 49.Lerner D, Adler DA, Rogers WH, Ingram E, Oslin DW. Effect of adding a work-focused intervention to integrated care for depression in the Veterans Health Administration: a randomized clinical trial. JAMA network open. 2020;3(2):e200075–e. 50. 50.Lobban F, Akers N, Appelbe D, Chapman L, Collinge L, Dodd S, et al. Clinical effectiveness of a web-based peer-supported self-management intervention for relatives of people with psychosis or bipolar (REACT): online, observer-blind, randomised controlled superiority trial. BMC psychiatry. 2020;20(1):1–16. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12888-020-02788-6&link_type=DOI) 51. 51.Lokkerbol J, Adema D, Cuijpers P, Reynolds III CF, Schulz R, Weehuizen R, et al. Improving the cost-effectiveness of a healthcare system for depressive disorders by implementing telemedicine: a health economic modeling study. The American Journal of Geriatric Psychiatry. 2014;22(3):253–62. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jagp.2013.01.058&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23759290&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 52. 52.Naveršnik K, Mrhar A. Cost-effectiveness of a novel e-health depression service. Telemedicine and e-Health. 2013;19(2):110–6. 53. 53.Osborne D, Meyer D, Moulding R, Kyrios M, Bailey E, Nedeljkovic M. Cost-effectiveness of internet-based cognitive-behavioural therapy for obsessive-compulsive disorder. Internet interventions. 2019;18:100277. 54. 54.Pil L, Pauwels K, Muijzers E, Portzky G, Annemans L. Cost-effectiveness of a helpline for suicide prevention. Journal of telemedicine and telecare. 2013;19(5):273–81. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1357633X13495487&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24163237&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 55. 55.thase ME, McCrone P, Barrett MS, Eells TD, Wisniewski SR, Balasubramani G, et al. Improving cost-effectiveness and access to cognitive behavior therapy for depression: providing remote-ready, computer-assisted psychotherapy in times of crisis and beyond. Psychotherapy and psychosomatics. 2020;89(5):307–13. 56. 56.Dukhovny D, Dennis C-L, Hodnett E, Weston J, Stewart DE, Mao W, et al. Prospective economic evaluation of a peer support intervention for prevention of postpartum depression among high-risk women in Ontario, Canada. American journal of perinatology. 2013;30(08):631–42. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23283805&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 57. 57.Joesch JM, Sherbourne CD, Sullivan G, Stein MB, Craske MG, Roy-Byrne P. Incremental benefits and cost of coordinated anxiety learning and management for anxiety treatment in primary care. Psychological medicine. 2012;42(9):1937–48. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1017/S0033291711002893&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22152230&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 58. 58.Kafali N, Cook B, Canino G, Alegria M. Cost-effectiveness of a randomized trial to treat depression among Latinos. The journal of mental health policy and economics. 2014;17(2):41. 59. 59.Simon GE, VonKorff M, Rutter C, Wagner E. Randomised trial of monitoring, feedback, and management of care by telephone to improve treatment of depression in primary care. Bmj. 2000;320(7234):550–4. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjAvNzIzNC81NTAiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8xMi8wMi8yMDIyLjEyLjAxLjIyMjgyODE3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 60. 60.Simon GE, Manning WG, Katzelnick DJ, Pearson SD, Henk HJ, Helstad CP. Cost-effectiveness of systematic depression treatment for high utilizers of general medical care. Archives of general psychiatry. 2001;58(2):181–7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/archpsyc.58.2.181&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11177120&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000166842600009&link_type=ISI) 61. 61.Wang PS, Patrick A, Avorn J, Azocar F, Ludman E, McCulloch J, et al. The costs and benefits of enhanced depression care to employers. Archives of General Psychiatry. 2006;63(12):1345–53. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/archpsyc.63.12.1345&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17146009&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000242492600005&link_type=ISI) 62. 62.Horn BP, Barragan GN, Fore C, Bonham CA. A cost comparison of travel models and behavioural telemedicine for rural, Native American populations in New Mexico. Journal of Telemedicine and Telecare. 2016;22(1):47–55. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1357633X15587171&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26026190&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 63. 63.topooco N, Berg M, Johansson S, Liljethörn L, Radvogin E, Vlaescu G, et al. Chat-and internet-based cognitive–behavioural therapy in treatment of adolescent depression: randomised controlled trial. BJPsych open. 2018;4(4):199–207. 64. 64.Yilmaz SK, Horn BP, Fore C, Bonham CA. An economic cost analysis of an expanding, multi-state behavioural telehealth intervention. Journal of telemedicine and telecare. 2019;25(6):353–64. 65. 65.Mlcoch T, Kruntorádová K, Mandelíková M, Dolezal T. Cost-Effectiveness Analysis of the Information Technology Aided Relaps Prevention Programme in Schizophrenia (Itareps) in the Czech Republic. Value in Health. 2015;18(7):A410. 66. 66.Chisholm D, Healey A, Knapp M. QALYs and mental health care. Social Psychiatry and Psychiatric Epidemiology. 1997;32(2):68–75. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/BF00788923&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9050347&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1997WJ17500004&link_type=ISI) 67. 67.McCrone P, Patel A, Knapp M, Schene A, Koeter M, Amaddeo F, et al. A comparison of SF-6D and EQ-5D utility scores in a study of patients with schizophrenia. J Ment Health Policy Econ. 2009;12(1):27–31. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19346564&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 68. 68.Aboujaoude E. Telemental health: why the revolution has not arrived. World Psychiatry. 2018;17(3):277. 69. 69.Mind. Digital services for people with mental health problems and digital exclusion during the coronavirus pandemic. Mind; 2020. 70. 70.Lee Y, Barendregt J, Stockings E, Ferrari A, Whiteford H, Patton G, et al. The population cost-effectiveness of delivering universal and indicated school-based interventions to prevent the onset of major depression among youth in Australia. Epidemiology and psychiatric sciences. 2017;26(5):545–64. 71. 71.Harley J. Economic evaluation of a tertiary telepsychiatry service to an island. J Telemed Telecare. 2006;12(7):354–7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1258/135763306778682378&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17059652&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 72. 72.Segal ZV, Dimidjian S, Beck A, Boggs JM, Vanderkruik R, Metcalf CA, et al. Outcomes of Online Mindfulness-Based Cognitive Therapy for Patients With Residual Depressive Symptoms: A Randomized Clinical Trial. JAMA Psychiatry. 2020;77(6):563–73. 73. 73.Health Quality Ontario. Internet-Delivered Cognitive Behavioural Therapy for Major Depression and Anxiety Disorders: A Health Technology Assessment. Ont Health Technol Assess Ser. 2019;19(6):1–199. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F12%2F02%2F2022.12.01.22282817.atom) 74. 74.Blankers M, Nabitz U, Smit F, Koeter MW, Schippers GM. Economic evaluation of internet-based interventions for harmful alcohol use alongside a pragmatic randomized controlled trial. Journal of medical Internet research. 2012;14(5):e2052. 75. 75.Abhulimen S, Hirsch A. Quantifying the economic impact of a digital self-care behavioral health platform on Missouri Medicaid expenditures. Journal of Medical Economics. 2018;21(11):1084–90. 76. 76.Lange K. Introducing Telepsychiatry into Routine Practice in an Emergency Department Psychiatric Service. Oxford Health NHS Foundation Trust; 2017.