A Digital Therapeutic Intervention for Smoking Cessation in Adult Smokers: Randomized Controlled Trial ====================================================================================================== * Jamie Webb * Sarrah Peerbux * Peter Smittenaar * Sarim Siddiqui * Yusuf Sherwani * Maroof Ahmed * Hannah MacRae * Hannah Puri * Sangita Bhalla * Azeem Majeed ## Abstract **Background** Tobacco smoking remains the leading cause of preventable death and disease worldwide. Digital interventions delivered through smartphones offer a promising alternative to traditional methods, but little is known about their effectiveness. **Objective** Our objective was to test the effectiveness of Quit Genius, a novel digital therapeutic intervention for smoking cessation. **Methods** A two-arm, single-blinded, parallel-group randomized controlled trial design was used. Participants were recruited via referrals from primary care practices and social media advertisements in the UK. 556 Adult smokers (aged ≥18 years), smoking at least five cigarettes a day for the past year were recruited. 530 were included for the final analysis. Participants were randomized to one of two interventions. Treatment consisted of a digital therapeutic intervention for smoking cessation consisting of a smartphone application delivering cognitive behavioral therapy content, one-to-one coaching, craving tools and tracking capabilities. The control intervention was Very Brief Advice along the Ask, Advise, Act model. All participants were offered nicotine replacement therapy for three months. A random half of each arm was assigned a carbon monoxide (CO) device for biochemical verification. Outcomes were self-reported via phone or online. The primary outcome was self-reported 7-day point prevalence abstinence at 4-weeks post quit date. **Results** 556 participants were randomized (treatment n=277, control n=279). The intention-to-treat analysis included 530 participants (n=265 in each arm; 11 excluded for randomization before trial registration, and 15 for protocol violations at baseline visit). By the quit date (an average 16 days after randomization) 89% (236/265) of those in the treatment arm were still actively engaged. At the time of primary outcome, 74% (196) of participants were still engaging with the app. At 4-weeks post-quit date, 45% (118) of participants in the treatment arm had not smoked in the preceding 7-days, compared to 29% (76) in control (risk ratio 1.55, 1.23-1.96, *P* = .0002; intention-to-treat, N=530). Self-reported 7-day abstinence agreed with CO measurement (CO <10 ppm) in 96% of cases (80/83) where CO readings were available. No harmful effects of the intervention were observed. **Conclusions** The Quit Genius digital therapeutic intervention is a superior treatment in achieving smoking cessation four weeks post quit date compared to very brief advice. **Trial Registration** The trial was registered in the ISRCTN database on December 18, 2018 ([https://www.isrctn.com/ISRCTN65853476](https://www.isrctn.com/ISRCTN65853476)). Key Words * Digital * Smoking * Cessation * Mobile * RCT * App ## Introduction Smoking is a leading cause of preventable and premature death worldwide. It is an important risk factor for serious health problems and life-threatening diseases [1]. Globally, tobacco use causes more than 8 million yearly deaths [2]. Moreover, the total global economic cost of smoking is more than $1.4 trillion a year [3]. Smoking is, therefore, a major worldwide economic and public health concern [1-3]. In the United Kingdom, Very Brief Advice (VBA) is the recommended clinical practice on smoking cessation for all health care practitioners [4-5]. It is designed to promote quit attempts and to be used opportunistically, in virtually any situation [4-5]. Those interested in quitting are referred to their local Stop Smoking Services (SSS) [4]. SSS typically combines face-to-face behavioral support with the option of pharmacotherapy, offered as nicotine replacement therapy (NRT) or varenicline. Similarly, in the United States, smokers access free telephone services and/or tobacco cessation websites that provide access to pharmacotherapy and additional behavioral support [6]. Despite traditional smoking cessation programs demonstrating efficacy, they only help ~15% of smokers to quit long-term [7]. Such programs have been shown to have limited utilization, due to scheduling, time and financial constraints [8-9]. Telephone support can overcome these barriers, but reaches only ~1% of smokers annually [10]. Given that support may be difficult to access, there is an urgent need for alternative solutions that are cost-effective, convenient and scalable. Technological advancements have led to new approaches that aim to overcome the drawbacks of conventional smoking cessation programs. Digital health apps are one new approach with the potential to support behavior change [11-12]. They have been used successfully across a multitude of therapeutic areas including chronic conditions [13-17] and to promote healthy behaviors [18-23]. Smartphone apps have advantages over traditional approaches including ease of accessibility, personalization of interactions with real-time feedback, scalability to large populations, and cost-effectiveness [24]. In 2018, the number of mobile phone subscriptions topped eight billion globally [24]. Smartphone apps have the potential to reach smokers who would not, or who are unable to, utilize traditional services. Yet, there is a paucity of data that examines the efficacy of smartphone apps for smoking cessation. A review of mobile phone-based smoking support identified only five randomized controlled trials (RCT) that tested the effectiveness of smartphone apps with low-intensity support, with each showing limited efficacy [24]. Furthermore, a recent content analysis revealed low adherence of existing smartphone apps to evidence-based treatment guidelines [25], whilst a review of the 50 most downloaded cessation apps found only two with scientific support [26]. Digital therapeutics are a new wave of digital health apps that can deliver high-intensity evidence-based therapeutic programs. Emerging evidence is encouraging and suggests that high-intensity support delivered via smartphone apps can aid smoking cessation. Yet, many early studies suffer from limitations such as utilizing single-arm cohorts or using non-CO verified self-reported abstinence to assess intervention efficacy [21-23]. To date, no RCT has been conducted on a high-intensity digital therapeutic intervention (DTI) for smoking cessation. ### Objectives This study had several objectives. The first objective of this study was to test the effectiveness of the DTI Quit Genius (QG) by measuring 7-day point prevalence abstinence at 4-weeks post quit date. Other objectives included assessing user engagement with QG, as well as testing its effect on cognitive, attitudinal, and emotional outcomes. ## Methods ### Design We conducted a two-arm, single-blinded, parallel-group, pre-registered randomized controlled trial, with 4-weeks, 6-months and 12-months follow-up. Here we report the primary outcomes at 4-weeks only as data collection for later time-points is still ongoing. Approval was granted by the Health and Social Care Research Ethics Committee A (HSC REC A; reference 18/NI/0171) and this research complies with the Declaration of Helsinki. ### Participants Participants were recruited in the United Kingdom between January and November 2019. Adult smokers (aged ≥18 years) were invited to participate if they had been smoking at least 5 cigarettes a day for the past year, were not using any other form of stop smoking support, and had sufficient mobile phone functionality (5th generation or higher for Apple iPhone or version 18 or higher for Android). The exclusion criteria included not speaking English, pregnancy, COPD, psychiatric medication, and a serious health condition that would substantially hinder completion of the intervention or control as determined by the study team. Participants with serious health conditions and/or using psychiatric medication were ruled out as a safety consideration. Participants were recruited offline from primary care practices across London via SMS campaigns. Posters and leaflets at local community venues and advertisements on social media were also used. Recruitment advertisements and study information given to participants described the opportunity of being allocated to one of two possible behavioral interventions in addition to optional NRT. No mention of a digital intervention was made to the participants before randomization. Participants randomized onto the study received £10 to offset travel expenses. Participants completed a questionnaire online or via telephone at 4-weeks after their quit-date and were paid £20 for completion. ### Registration The trial was registered in the ISRCTN database on December 18, 2018 ([https://www.isrctn.com/ISRCTN65853476](https://www.isrctn.com/ISRCTN65853476)). On July 24, 2019, we adjusted the primary outcome to equate it with the majority of trials on smoking cessation. Specifically, the time window for assessing whether the participant had successfully quit was reduced from two weeks to one week. The previously approved primary outcome related to the Russell Standard (i.e. self-reported abstinence in the past 2-weeks at 4-weeks post quit date). This was changed to capture self-reported 7-day point prevalence abstinence at 4-weeks. This decision was informed in-part by a recent Delphi study which found only partial compliance with the Russell Standard, as reported by smoking cessation experts [27]. Although we acknowledged a lack of consensus relating to outcome criteria in smoking cessation research, it was of our opinion that the use of 7-day point prevalence is preferable to 14-day point prevalence as it allows for greater comparability to other studies. Amending the primary outcome to 7-day point-prevalence allowed for greater comparison with other studies of face-to-face, digital and other low intensity intervention smoking cessation trials(full justification is available on the ISRCTN page). This decision was made without having analyzed trial outcomes. The 14-day point prevalence was added as a secondary outcome. Another secondary outcome (“number of quit attempts up to week four post-quit date”) is reported here as “Any additional quit attempt after the quit date” because a continuous test was not appropriate on the distribution which was predominantly one quit attempt. All other adjustments to the trial registration only concerned start/end dates of recruitment and publication to account for unexpected challenges in recruitment. ### Randomization and masking Participants were randomized 1:1 (treatment:control) using a block size of four participants through a trial management system ([www.curebase.com](http://www.curebase.com)). Researchers randomizing participants were blind to allocation until they had performed the randomization. ### Procedures At first contact (via phone or online), participants were provided with study information and completed a questionnaire to determine eligibility. If eligible, participants were invited to attend an in-person baseline session where eligibility was reconfirmed and informed consent and baseline data was collected. Participants were then randomized. All participants were recommended to set their quit date within two-weeks of randomization, however this was not a mandatory requirement for study participation. All participants were offered NRT in addition to their allocated intervention. At 4-weeks post-quit-date participants were invited to complete the follow-up survey (via phone or online). ### Treatment intervention: Quit Genius Quit Genius (QG) is a smartphone application comprising a comprehensive digital solution informed by the principles of cognitive behavior therapy (CBT) (Screenshots shown in Figure 1). It is a year-long program designed to support the user both before and after their quit date. Quit Genius delivers intervention components that have demonstrated efficacy in promoting smoking cessation, including self-monitoring, goal setting, encouraging medication adherence, and feedback on progress ([www.quitgenius.com](http://www.quitgenius.com)). QG was developed over many iterations including engagement with smokers, patient representatives and scientific advisors. The application collects data on users through in-app metrics to help personalize the program. Metrics include usage, session completion, program completion and quit date. Additional data are collected based on user participation and feedback following CBT exercises, giving information such as their reasons for quitting smoking and the reasons why they continue to smoke. ![Figure 1:](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2020/08/19/2020.06.25.20139741/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2020/08/19/2020.06.25.20139741/F1) Figure 1: Screenshots of the digital therapeutic intervention Quit Genius Content is tailored to the user and is delivered in the form of animated videos, audio sessions, reflective exercises, and quizzes. The user is prompted to complete a series of self-paced steps on their smoking cessation journey, with each new step (and content) unlocking only once the previous step has been completed. The program content is divided into the ‘Essentials’ stage, which the user is prompted to complete before their quit-date, and covers aspects such as preparing for the quit date, using nicotine replacement therapy, and thinking about the reasons for quitting. The ‘Sustain’ stage, which the user completes once they quit smoking, focuses on the general principles of relapse prevention and helps the user to stay smoke-free in the long-term. In the time leading up to their quit date, users are encouraged to monitor their smoking habits daily by logging the number of cigarettes smoked, their triggers (how they felt when they wanted to smoke), and the intensity of their craving. Once the user has quit smoking, they are encouraged to log whether or not they are currently smoking. As part of the QG program participants also have access to a ‘Quit Coach’, an advisor qualified by the National Centre for Smoking Cessation and Training (NCSCT). The coach provides personalized support via a digital chat interface and phone. Typically, participants partake in one initial phone call with the rest of quit coach interaction mediated through the in-app digital chat interface. Users can monitor their progress via the app, which details improvements to health and any financial benefits gained from being a non-smoker since their quit date. Finally, users can access a ‘Craving Toolbox’ which comprises audio content of short breathing exercises, mindfulness exercises and meditation exercises designed to help the user manage their cravings to smoke. The QG app uses CBT to target not only smoking cessation but also covers skills and strategies to promote improved mental health and wellbeing. Whilst the QG app does not substitute professional care for mental health concerns, the app specifically addresses common mental health concerns such as low mood, anxiety, stress, self-esteem, and social skills. The app also specifically targets general health and wellbeing concerns such as diet, exercise, and self-care techniques. Specific skills and techniques used include goal setting, cognitive restructuring, graded exposure, progressive muscle relaxation, mindfulness, assertiveness and communication training, and problem-solving skills. QG users receive push notifications to serve as reminders to engage with the app. All participants in the treatment group received free access to the QG intervention. ### Control group: Very Brief Advice VBA is a simple form of advice designed to be used opportunistically. It follows the Ask, Advise, Act structure as recommended by the UK government. Participants were advised to contact their local stop smoking service to access support and medication to quit smoking. Trial assistants were trained in the delivery of VBA as per NCSCT guidelines ([https://elearning.ncsct.co.uk](https://elearning.ncsct.co.uk)). Of the control group participants allocated a CO device, a non-branded mobile-app (ASH-app) was also provided to visualize CO readings for the participant. The control group mobile-app was only used in conjunction with the CO device and contained no other content for participants. ### Nicotine Replacement Therapy All participants had the option to receive nicotine replacement products (2mg or 4mg gum and/or 16hr or 24hr patches) free for 12 weeks, with the first two-week supply issued at the baseline visit. Participants were allowed to purchase alternative forms of oral NRT. ### Carbon Monoxide Monitor Half of all participants were given a carbon monoxide (CO) monitor to measure levels of carbon monoxide in their breath and to validate self-reported smoking abstinence. Participants were selected pseudorandomly to ensure 50% of each group was assigned a device. CO levels were collected via self-report at 4-weeks post quit-date. The CO devices plugged into the headphone/charging slot of participants’ smartphones, and were used in conjunction with the QG and control app. At the follow-up time-point, participants were asked to give a reading from their device via phone or online. NCSCT guidelines of a CO reading <10ppm were used to validate participants’ self-reported abstinence [28]. ### Measures Measurements were taken at baseline and at the 4-week follow-up. The following variables were collected at baseline: demographic details, smoking status, smoking history, expired carbon monoxide level, Fagerstrom Test for Nicotine Dependence (FTND) [29], Smoking Abstinence Self-efficacy Questionnaire (SASEQ) [30], Warwick-Edinburgh Mental Well-being Scale (WEMWBS) [31], WHOQOL-BREF [32] and the Service Use Questionnaire (SUQ) [33]. At 4-weeks post quit-date, smoking status, changes in attitudes and perceptions of smoking, SASEQ, WEMWBS, and WHOQOL-BREF were collected. Expired carbon monoxide level was collected only in those participants who were assigned their own device (Smokerlyzer, coVita Inc.). Measurements were collected via online questionnaires. The primary outcome was self-reported 7-day point prevalence abstinence at 4-weeks post quit-date. Secondary outcomes at week four included 14-day point prevalence abstinence, any additional quit attempts after the quit-date, self-reported changes in confidence levels, knowledge, attitudes and perceptions related to smoking cessation, changes in SASEQ and WEMWBS, and satisfaction with the treatment intervention (treatment group only). ### Data Analysis At 4-weeks, we expected to observe a 7-day abstinence rate of 25% in the treatment group and 10% in the control group. At a Type I error rate of 5% and power of 90% we required 133 participants per group (266 total). At 6-months, a conservative estimate would be a 10% quit rate in the treatment group and 3% in the control group. To detect a difference with 80% power and 5% Type I error we needed to randomize 194 participants per group (388 total). Assuming a 20% dropout, we aimed to recruit ≥500 participants. We performed both intention-to-treat (ITT) [34] and per-protocol (PP) analyses. ITT included all participants assigned to treatment and control, respectively, as indicated in Figure 1. ITT analysis assumed that participant data was not missing at random. PP included the subset of participants that provided answers to the self-reported outcomes at week-4. We used chi-squared tests for binary outcomes and two-sample t-tests for continuous outcomes. Those lost to follow-up at 4-weeks were considered “currently smoking” for the primary outcome as well as 2-week abstinence; no additional quit attempts after the quit date; not “strongly agree” to improvements in confidence, knowledge, or attitude; and no change in self-efficacy or mental well-being from baseline. We used logistic regression to estimate the main effect of being assigned a CO device on likelihood of quitting, with treatment assignment as a covariate in the model. All data processing and analysis was performed in R using the *tidyverse* package family and the *fmsb* package [35–37]. ## Results ### Participant flow chart Figure 2 shows the CONSORT flowchart for the RCT. ![Figure 2:](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2020/08/19/2020.06.25.20139741/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2020/08/19/2020.06.25.20139741/F2) Figure 2: CONSORT flow chart ### Participants 556 participants were randomized (treatment n=277, control n=279). The intention-to-treat analysis included 530 participants (n=265 in each arm; 11 excluded for randomization before trial registration, and 15 for protocol violations at baseline visit). Participants came from a wide age range, were slightly more likely to be male than female, and about two in three self-identified as Caucasian/White (Table 1). Educational attainment ranged uniformly from secondary education up to postgraduate education, and 80.0% (424/530) were in paid employment, of which over half were in “Managerial or professional” roles. Participants were smoking on average 14 (treatment) or 15 (control) cigarettes per day, with nicotine dependence of four out of 10 on the Fagerstrom Test for Nicotine Dependence. Most participants (85.0%, 451/530) had previously made quit attempts, primarily going cold turkey or with the help of e-cigarettes and NRT. No substantial differences between treatment and control groups were introduced through randomization. View this table: [Table 1:](http://medrxiv.org/content/early/2020/08/19/2020.06.25.20139741/T1) Table 1: Demographics and smoking history of treatment and control groups. SD: standard deviation. NRT: nicotine replacement therapy ### Engagement Engagement with different facets of the DTI is shown in Table 2. By the quit date (an average 16 days after randomization) 89.0% (236/265) of those in the treatment arm were still actively engaged. At the time of primary outcome, 74.0% (196) of participants were still engaging with the app. The content consisting of education and cognitive behavioral therapy (“Essentials” 1 and 2) were consumed by 55.0% (146) and 36% (95) of participants, respectively. 69.0% (183) sent at least one in-app message to their coach, and on average people messaged their coach about once per week. On average, participants reported 12 diary entries to report cigarettes smoked before their quit date, and six check-ins to report cravings or lapses after their quit date. View this table: [Table 2:](http://medrxiv.org/content/early/2020/08/19/2020.06.25.20139741/T2) Table 2: Engagement with the digital therapeutic program in the treatment group (intention-to-treat, n = 265). Essentials 1: program content aimed at preparation for the quit date. Essentials 2: program content intended for just after the quit date. Diary entry: registration of a cigarette smoked prior to quit date. Check-in: registration of a cigarette after the quit date ### Outcomes Table 3 shows the primary outcome and secondary outcomes for intention-to-treat and per-protocol analyses. In the intention-to-treat analysis, those in the treatment arm were 55% more likely to report 7-day abstinence 4-weeks after their quit date compared to those in control (RR: 1.55, 95% CI 1.23-1.96; 45% versus 29% quit rate). In those participants that were assigned a CO device (138/265 in treatment, 142/265 in control), 97% (134) in treatment and 98% (139) in control provided a CO reading at baseline, and 61% (82) and 67% (95) respectively provided a reading at 4-weeks (including those that did not complete the week-4 questionnaire). CO completion was 88% (50/57) and 89% (33/37) respectively in participants that claimed abstinence at 4-weeks. For these abstaining participants the CO measurement was below 10 parts per million (ppm) for 96% (48/50) and 97% (32/33) of participants in treatment and control, respectively. Whether or not a participant was provided with a CO device did not significantly predict quit rate (p=.29 in logistic regression with CO device and intervention main effects). There was no difference in rate of NRT use in treatment and control (59.0% and 63.0% respectively in those that completed week-4 questionnaire; RR: 0.94, 95% CI 0.81-1.08) View this table: [Table 3:](http://medrxiv.org/content/early/2020/08/19/2020.06.25.20139741/T3) Table 3: Outcomes at four weeks after quit date. ITT: intention-to-treat. PP: per protocol. WEMWBS: Warwick-Edinburgh Mental Wellbeing Scale. SASEQ: Smoking abstinence self-efficacy questionnaire. * as measured by % of participants reporting “strongly agree” For secondary outcomes, 14-day abstinence showed greater efficacy of treatment compared to control (RR: 1.47, 95% CI 1.12-1.93). Those in treatment were no more or less likely to have made an additional quit attempt after the initial quit date (RR: 0.79, 95% CI 0.60-1.03). Those in the treatment arm were more likely to “strongly agree” that their knowledge of their smoking habit had improved (RR: 1.35, 95% CI 1.06-1.72), but no such effect was observed regarding their confidence in their ability to stay smoke-free (RR: 1.29, 95% CI 0.98-1.70]) nor in terms of whether their attitude towards stopping smoking had become more positive (RR: 0.97, 95% CI 0.78-1.21). The treatment was also not superior to control in terms of the increase in smoking self-efficacy (*P* = .09), nor in terms of mental well-being (P = .83). However, in the per-protocol analysis, which included only participants that completed their week-4 outcomes, several secondary outcomes were significantly better in treatment compared to control: confidence improved more (RR: 1.33, 95% CI 1.02-1.74), as did self-efficacy (P = .04). Participant satisfaction with the treatment intervention was high (209/265 in the treatment group that completed the questionnaire at week four). On a scale from zero (least satisfied) to three (most satisfied): quality of the smoking cessation service: 2.4, SD 0.7; program meeting needs: 2.2, SD 0.8; helpfulness of information: 2.6, SD 0.6; helped to deal with smoking effectively: 2.4, SD 0.7; likelihood of coming back if needing to quit in the future: 2.5, SD 0.7. Asked whether the participant would recommend the digital therapeutic to a friend, 92% (192) would do so. Participants considered the quit coach and the education/CBT articles to be most helpful, whereas the community element of the program was considered least helpful (as shown in Figure 3). ![Figure 3:](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2020/08/19/2020.06.25.20139741/F3.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2020/08/19/2020.06.25.20139741/F3) Figure 3: Elements of the program considered most helpful by n=209 participants who completed the user satisfaction questionnaire at week four Finally, we examined two outcomes unregistered at trial registration. First, cigarettes smoked per day by those that failed to quit showed no difference between treatment and control (treatment: 7.9, control: 7.5 cigarettes per day). Similarly, the average % reduction in cigarettes per day, though substantial in both groups, showed no difference between groups (48.1% reduction in treatment vs 48.9% in control; *P* = .83). ## Discussion ### Principal findings In this RCT we assessed the efficacy of Quit Genius, a comprehensive DTI for smoking cessation. The primary outcome of self-reported 7-day abstinence at 4-weeks post quit date was significantly higher for the treatment group compared to a control group using VBA. Principally, this shows the superiority of the DTI compared to the UK’s typical first-line intervention for smoking cessation [4-5]. We show that a digital, remote-first program is an effective method for short-term behavior change. The treatment group demonstrated a CO-verified 4-week quit rate comparable with high-intensity face-to-face smoking cessation programs utilized by healthcare services in the United Kingdom [7]. Our results support previous literature illustrating that high-intensity behavioral support combined with pharmacotherapy is an effective means of quitting smoking [7]. Compared with other digital interventions, the 45.0% (ITT) and 52.0% (PP) seven-day abstinence rates four weeks after the quit date compare favorably to previous digital intervention studies. Pivot, another digital program, achieved a CO-verified benchmark seven-day abstinence rate of 32% (ITT) and 37% (PP) [21]. An Acceptance and Commitment Therapy (ACT) intervention by Smartquit achieved 21% 7-day abstinence [22]. Clickotine’s 7-day abstinence rate of 45% was similar to that observed in the current study [23]. However, all used single-arm designs rather than RCTs, and were therefore unable to distinguish between the causal effect of treatment, placebo, or underlying differences in propensity to quit in the study population [38]. Finally, both Smartquit and Clickotine studies used non-CO-verified self-reported abstinence as their primary means of assessing intervention efficacy, leaving a possibility of falsely reported smoking status [22-23]. This study avoided such limitations by using a two-arm, parallel-group RCT design with biochemical verification. Though short-term abstinence rates of the program studied here are promising both in absolute terms as well as compared to the control group, 6- and 12-month abstinence rates will confirm whether the program is efficacious in the longer term. ### Cognitive, attitudinal, and emotional improvements Education and confidence are integral mechanisms in eliciting successful smoking cessation [39-40]. In this study, treatment caused a greater improvement in knowledge of personal smoking habits compared to control. Similarly, confidence in ability to stay smoke-free was higher in treatment than control, though only statistically significantly so in participants that completed the study per protocol. No effect was found on the attitude of participants towards stopping smoking. This suggests that the DTI is an effective tool for making people aware of their habit and instilling some degree of confidence, but fails to improve a commonly negative attitude towards stopping smoking. Given that smoking cessation interventions may be enhanced by incorporating strategies that target attitude change [41], the DTI could be further improved to engender a more positive attitude towards quitting. Self-efficacy is a robust indicator of future successful smoking abstinence (40). We observed the DTI to be superior to control in improving self-efficacy in participants that completed the study per protocol, but not when analyzed by intention-to-treat. This resembles the increase in reported confidence, indicating that the DTI enhanced users’ beliefs in their capacity to quit successfully but that further developments on the DTI should focus on strengthening these outcomes. We observed no benefit of the DTI compared to control in terms of mental well-being. In a retrospective study of the same DTI we observed a correlation between hedonic well-being and quit rates [42]. In line with this, smoking cessation is typically associated with improved mental health with evidence illustrating a reduction in anxiety, depression and stress after quitting [43]. Given the higher quit rates in the treatment group, we expected treatment to be superior in terms of mental well-being. However, improvements are usually demonstrated in longer-term follow-ups than the four weeks reported here [43]. As such, changes in mental well-being might not yet have manifested, and our 6-month and 12-month outcomes will shed light on the longer-term impact of the DTI on mental well-being. There are both positive and negative effects associated with smoking and smoking abstinence that could impact mental well-being. Evidence suggests that smokers who reduce their smoking but fail to quit show more pronounced mood deterioration than those who succeed [43]. Conversely, there is evidence illustrating the negative sequelae of smoking cessation, such as anxiety, insomnia and weight gain [44]. Therefore, it is possible that simultaneous effects could have been active within groups, resulting in no net effect on mental well-being. Nicotine addiction is a condition that rarely exceeds a lifetime abstinence rate of >50% pertreatment. For this reason, many smokers take 30 or more quit attempts before being successful [45]. Nonetheless, a reduction in daily cigarette use predicts future behavior change: individuals who reduce their cigarette use by ~50% are more likely to see future quit attempts and greater odds of successful quitting [45]. Among treatment participants who had not quit smoking, there was an average reduction of 48% in their cigarette use compared to baseline, similar to control. Digital programs can, perhaps more easily than traditional programs, leverage such data to continue to engage and encourage specific participants after a failed quit attempt, identifying an optimal time to engage when chances of quitting are highest. ### Engagement To elicit the success of any non-invasive behavioral intervention, users must actively partake in the treatment [17, 23]. We assessed engagement across several elements of the DTI, and observed 74% (196/265) were still using the app four weeks after their quit date. One differentiating aspect of this DTI from typical smartphone applications is the presence of human coaching. We found a consistent bidirectional flow of communication between quit coach and participant from baseline to quit date and from quit date to 4-week follow-up. Of all the elements of treatment offered by the DTI, coaching was considered most helpful. This reflects previous notions that smoking cessation programmes that utilize health coaching as a means of support are effective in eliciting successful smoking abstinence [46]. ### Strengths and limitations Particular strengths of this study are the randomized controlled design, preregistration of the trial and its outcomes, biochemical verification despite the remote nature of the program, and the high four-week follow-up rate. The key limitation of this study is the short follow-up period of 4-weeks. It is known that relapse occurs over a longer timeframe [47-49], and the current findings do not speak to the DTI’s ability to avoid longer-term relapse. Another limitation is that participants may have exaggerated their self-reported smoking abstinence. To combat this, participants were informed from study outset that regardless of smoking abstinence, they could remain on the trial and would be eligible for remuneration. Another preventative measure was the use of a measurement device to assess CO levels in the exhaled breath of the individual. Due to cost considerations, these devices were provided to a random 50% of each study arm. The near-perfect agreement (80/83, 96%) between CO monitor and self-reported abstinence suggests that at least in the context of this DTI, self-report can be taken at face value. Lastly, the lack of experimenter blinding to participant group allocation may have introduced bias into data interpretation. To avoid this, standardized questionnaires, participant interaction scripts, and standard operating procedures were used across treatment groups, so that any effect on participant outcome data was minimized. Also, the trial outcome measures, and sample size were preregistered, and data was only analysed after data collection had been completed. ### Conclusions A digital therapeutic intervention for smoking cessation was superior to very brief advice in achieving smoking cessation after 4-weeks in a biochemically verified RCT. The DTI examined here is an effective option for short-term smoking cessation. Participants were actively engaged with the DTI and were satisfied with the intervention. Nevertheless, opportunities exist to improve mental wellbeing and attitudinal outcomes. A critical open question pertains to the long-term efficacy, which will be reported in a subsequent paper. ## Data Availability Data is available upon request ## Conflict of Interests The study was funded by the company that produced the Quit Genius digital therapeutic intervention (Digital Therapeutics, Inc). PS is a paid statistical consultant. All other authors except AM received a salary from or own equity in Digital Therapeutics, Inc. Abbreviations : ACT : acceptance and commitment therapy CBT : cognitive behavioral therapy CO : carbon monoxide DTI : digital therapeutic intervention ITT : intention-to-treat NRT : nicotine replacement therapy PP : per-protocol QG : quit genius RCT : randomized controlled trial VBA : very brief advice ## Acknowledgements The authors thank the participating NHS primary care practices located across the London districts of Barnet, Camden, Ealing, Hammersmith and Fulham, Islington, Kilburn and North Kensington for their support in participant recruitment. AM is supported by the NW London NIHR Applied Research Collaboration. ## Footnotes * Clinical trial registration: ISRCTN registry, number [ISRCTN65853476](http://medrxiv.org/external-ref?link_type=ISRCTN&access_num=ISRCTN65853476) ([https://www.isrctn.com/ISRCTN65853476](https://www.isrctn.com/ISRCTN65853476)) * Received June 25, 2020. * Revision received August 19, 2020. * Accepted August 19, 2020. * © 2020, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. 1.Organization WH. World Health Organization Report on the Global Tobacco Epidemic, 2011: warning about the dangers of tobacco. 2011. World Health Organization: Geneva, Switzerland. 2011. 2. 2.Organization WH. WHO report on the global tobacco epidemic 2019: Offer help to quit tobacco use. 2019. World Health Organization: Geneva, Switzerland. 2019. 3. 3.Goodchild M, Nargis N, d’Espaignet ET. Global economic cost of smoking-attributable diseases. Tob Control. 2018;27(1):58–64. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6InRvYmFjY29jb250cm9sIjtzOjU6InJlc2lkIjtzOjc6IjI3LzEvNTgiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMC8wOC8xOS8yMDIwLjA2LjI1LjIwMTM5NzQxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 4. 4.National Institute for Health and Care Excellence (NICE) (2013). Smoking cessation: supporting people to stop smoking. Quality Standard and Indicators. London: NICE. Available at [https://www.nice.org.uk/guidance/qs43/documents/smoking-cessation-supporting-people-to-stop-smoking-briefingpaper2](https://www.nice.org.uk/guidance/qs43/documents/smoking-cessation-supporting-people-to-stop-smoking-briefingpaper2). [Accessed 2020 Mar 27th]. 5. 5.McEwen A, Pooler J, Lionis C, Papadakis S, Tsiligianni I, Anastasaki M, et al. Adapting Very Brief Advice (VBA) on smoking for use in low-resource settings: experience from the FRESH AIR project. J Smok Cessat. 2019;14(3):190–194. 6. 6.Centers for Disease Control and Prevention. 2014. [2020-04-10]. Best Practices for Comprehensive Tobacco Control Programs—2014: Cessation Interventions [https://www.cdc.gov/tobacco/stateandcommunity/best\_practices/pdfs/2014/sectionA-III.pdf](https://www.cdc.gov/tobacco/stateandcommunity/best_practices/pdfs/2014/sectionA-III.pdf) 7. 7.Bauld L, Bell K, McCullough L, Richardson L, Greaves L. The effectiveness of NHS smoking cessation services: a systematic review. J Public Health. 2009 Jul 28;32(1):71–82. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19638397&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000276156600015&link_type=ISI) 8. 8.Kotz D, Fidler J, West R. Factors associated with the use of aids to cessation in English smokers. Addiction. 2009;104(8):1403–1410. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1360-0443.2009.02639.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19549267&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000268028000024&link_type=ISI) 9. 9.Patrick H, Fujii CA, Glaser DB, Utley DS, Marler JD. A comprehensive digital program for smoking cessation: assessing feasibility in a single-group cohort study. JMIR MHealth UHealth. 2018;6(12):e11708. 10. 10.Best Practices for Comprehensive Tobacco Control Programs: 2014. Tob Control. 2014;144. 11. 11.Meier E, Tackett AP, Wagener TL. Effectiveness of electronic aids for smoking cessation. Curr Cardiovasc Risk Rep. 2013;7(6):464–472. 12. 12.Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. 2016 Nov;51(5):843–51. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) 13. 13.Pal K, Eastwood SV, Michie S, Farmer AJ, Barnard ML, Peacock R, et al. Computer-based diabetes self-management interventions for adults with type 2 diabetes mellitus. Cochrane Database Syst Rev. 2013;(3). 14. 14.Zhuo X, Zhang P, Kahn HS, Bardenheier BH, Li R, Gregg EW. Change in medical spending attributable to diabetes: national data from 1987 to 2011. Diabetes Care. 2015;38(4):581–587. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZGlhY2FyZSI7czo1OiJyZXNpZCI7czo4OiIzOC80LzU4MSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzA4LzE5LzIwMjAuMDYuMjUuMjAxMzk3NDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 15. 15.Murray E, Burns J, Tai SS, Lai R, Nazareth I. Interactive Health Communication Applications for people with chronic disease. Cochrane Database Syst Rev. 2005;(4). 16. 16.Peyrot M, Rubin RR, Funnell MM, Siminerio LM. Access to diabetes self-management education. Diabetes Educ. 2009;35(2):246–263. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0145721708329546&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19208816&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000264637000007&link_type=ISI) 17. 17.Shebib R, Bailey JF, Smittenaar P, Perez DA, Mecklenburg G, Hunter S. Randomized controlled trial of a 12-week digital care program in improving low back pain. NPJ Digit Med. 2019;2(1):1–8. 18. 18.Harris J, Felix L, Miners A, Murray E, Michie S, Ferguson E, et al. Adaptive e-learning to improve dietary behaviour: a systematic review and cost-effectiveness analysis. Health Technol Assess Winch Engl. 2011;15(37):1–160. 19. 19.Khadjesari Z, Murray E, Hewitt C, Hartley S, Godfrey C. Can stand-alone computer-based interventions reduce alcohol consumption? A systematic review. Addiction. 2011;106(2):267–282. 20. 20.Bailey JV, Murray E, Rait G, Mercer CH, Morris RW, Peacock R, et al. Interactive computer-based interventions for sexual health promotion. Cochrane Database Syst Rev. 2010;(9). 21. 21.Marler JD, Fujii CA, Utley DS, Tesfamariam LJ, Galanko JA, Patrick H. Initial assessment of a comprehensive digital smoking cessation program that incorporates a mobile app, breath sensor, and coaching: cohort study. JMIR MHealth UHealth. 2019;7(2):e12609. 22. 22.Iacoviello BM, Steinerman JR, Klein DB, Silver TL, Berger AG, Luo SX, et al. Clickotine, a personalized smartphone app for smoking cessation: initial evaluation. JMIR MHealth UHealth. 2017;5(4):e56 23. 23.Bricker JB, Mull KE, Kientz JA, Vilardaga R, Mercer LD, Akioka KJ, et al. Randomized, controlled pilot trial of a smartphone app for smoking cessation using acceptance and commitment therapy. Drug Alcohol Depend. 2014;143:87–94. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.drugalcdep.2014.07.006&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25085225&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) 24. 24.Whittaker R, McRobbie H, Bullen C, Rodgers A, Gu Y, Dobson R. Mobile phone text messaging and app-based interventions for smoking cessation. Cochrane Database Syst Rev. 2019;(10). 25. 25.Abroms LC, Westmaas JL, Bontemps-Jones J, Ramani R, Mellerson J. A content analysis of popular smartphone apps for smoking cessation. Am J Prev Med. 2013;45(6):732–736. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.amepre.2013.07.008&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24237915&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) 26. 26.Haskins BL, Lesperance D, Gibbons P, Boudreaux ED. A systematic review of smartphone applications for smoking cessation. Transl Behav Med. 2017;7(2):292–299. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s13142-017-0492-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=28527027&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) 27. 27.Cheung KL, de Ruijter D, Hiligsmann M, Elfeddali I, Hoving C, Evers SMAA, et al. Exploring consensus on how to measure smoking cessation. A Delphi study. BMC Public Health. 2017 Nov 21;17(1):890. 28. 28.Shahab L. Why use CO-verified 4-week quit rates as the primary measure of stop smoking service success? 2014; 29. 29.Heatherton TF, Kozlowski LT, Frecker RC, Fagerstrom K-O. The Fagerstrom test for nicotine dependence: a revision of the Fagerstrom Tolerance Questionnaire. Br J Addict. 1991;86(9):1119–1127. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1360-0443.1991.tb01879.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=1932883&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1991GD27400011&link_type=ISI) 30. 30.Spek V, Lemmens F, Chatrou M, van Kempen S, Pouwer F, Pop V. Development of a smoking abstinence self-efficacy questionnaire. Int J Behav Med. 2013;20(3):444–449. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) 31. 31.Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick-Edinburgh mental well-being scale (WEMWBS): development and UK validation. Health Qual Life Outcomes. 2007;5(1):63. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1477-7525-5-63&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18042300&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) 32. 32.Group W. Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychol Med. 1998;28(3):551–558. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1017/S0033291798006667&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9626712&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000073793600006&link_type=ISI) 33. 33.EQUIP Service Use Questionnaire: DIRUM - Database of Instruments for Resource Use Measurement [Internet]. Available from: [http://www.dirum.org/instruments/details/111#](http://www.dirum.org/instruments/details/111#) [Accessed 2020 Jan 17]. 34. 34.West R, Hajek P, Stead L, Stapleton J. Outcome criteria in smoking cessation trials: proposal for a common standard. Addict Abingdon Engl. 2005 Mar;100(3):299–303. 35. 35.Team RC. R: A language and environment for statistical computing. Vienna, Austria; 2013. 36. 36.Wickham H, Averick M, Bryan J, Chang W, McGowan LD, François R, et al. Welcome to the Tidyverse. J Open Source Softw. 2019;4(43):1686. 37. 37.Nakazawa M. fmsb: Functions for Medical Statistics Book with some Demographic Data [Internet]. 2019 [cited 2020 Aug 5]. Available from:[https://CRAN.R-project.org/package=fmsb](https://CRAN.R-project.org/package=fmsb) 38. 38.Evans SR. Clinical trial structures. J Exp Stroke Transl Med. 2010 Feb 9;3(1):8–18. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21423788&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) 39. 39.Golechha M. Health Promotion Methods for Smoking Prevention and Cessation: A Comprehensive Review of Effectiveness and the Way Forward. Int J Prev Med [Internet]. 2016 Jan 11;7. Available from: [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4755211/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4755211/) [Accessed 2020 Apr 2]. 40. 40.Gwaltney CJ, Metrik J, Kahler CW, Shiffman S. Self-Efficacy and Smoking Cessation: A Meta-Analysis. Psychol Addict Behav J Soc Psychol Addict Behav [Internet]. 2009 Mar;23(1). Available from: [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3829471/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3829471/) [Accessed 2020 Apr 2]. 41. 41.Lee HS, Addicott M, Martin LE, Harris KJ, Goggin K, Richter KP, et al. Implicit attitudes and smoking behavior in a smoking cessation induction trial. Nicotine Tob Res. 2017;20(1):58–66. 42. 42.Lin Y, Tudor-Sfetea C, Siddiqui S, Sherwani Y, Ahmed M, Eisingerich AB. Effective behavioral changes through a digital mHealth app: exploring the impact of hedonic wellbeing, psychological empowerment and inspiration. JMIR MHealth UHealth. 2018;6(6):e10024. 43. 43.Taylor G, McNeill A, Girling A, Farley A, Lindson-Hawley N, Aveyard P. Change in mental health after smoking cessation: systematic review and meta-analysis. Bmj. 2014;348:g1151. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDgvZmViMTNfMS9nMTE1MSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzA4LzE5LzIwMjAuMDYuMjUuMjAxMzk3NDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 44. 44.Hajek P. Introduction: Recognition and management of some possible consequences of stopping smoking. Taylor & Francis; 2009. 45. 45.Chaiton M, Diemert L, Cohen JE, Bondy SJ, Selby P, Philipneri A, et al. Estimating the number of quit attempts it takes to quit smoking successfully in a longitudinal cohort of smokers. BMJ Open. 2016 Jun 1;6(6):e011045. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiNi82L2UwMTEwNDUiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMC8wOC8xOS8yMDIwLjA2LjI1LjIwMTM5NzQxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 46. 46.Sforzo GA, Kaye M, Ayers GD, Talbert B, Hill M. Effective tobacco cessation via health coaching: an institutional case report. Glob Adv Health Med. 2014;3(5):37–44. 47. 47.Herd N, Borland R, Hyland A. Predictors of smoking relapse by duration of abstinence: findings from the International Tobacco Control (ITC) Four Country Survey. Addiction. 2009;104(12):2088–2099. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1360-0443.2009.02732.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19922574&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000271625500019&link_type=ISI) 48. 48.Brandon TH, Lazev AB, Juliano LM. Very delayed smoking relapse warrants research attention. Psychol Rep. 1998;83(1):72–74. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2466/pr0.1998.83.1.72&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9775662&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) 49. 49.Wetter DW, Cofta-Gunn L, Fouladi RT, Cinciripini PM, Sui D, Gritz ER. Late relapse/sustained abstinence among former smokers: a longitudinal study. Preventive Medicine. 2004 Dec 1;39(6):1156–63. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ypmed.2004.04.028&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15539050&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F19%2F2020.06.25.20139741.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000225496200014&link_type=ISI)