Study Protocol for a Research and Development Project: Optimizing a Unified Parent Training Intervention to Prevent Child Mental Health Problems and Neglect

Needs for Innovation

Despite several decades of development and evaluation of evidence-based interventions (EBIs) to promote positive child development, several interrelated challenges concerning EBIs remain. EBIs continue to exhibit small to moderate effect sizes (Duncan & Magnuson, 2013; Supplee & Duggan, 2019; Weisz et al., 2017), and few children and families have access to interventions that have been documented as effective (Christiansen, 2015; Glasgow et al., 2012; Nøkleby, Johansen, Jardim, & Muller, 2020; Skogen & Torvik, 2013). Thus, the public health impact of EBIs is probably low. Moreover, the nature of mental health symptoms often transcends traditional symptom domains, and comorbidity of mental health problems in children is very common (Caspi et al., 2020). Children often display both internalizing and externalizing symptoms (Caspi et al., 2020; Marchette & Weisz, 2017). The traditional assumption that the identification of one mental health diagnosis or problem identifies the root cause of the problem and thus indicates the manner in which treatment should be delivered is a poor match with the comorbid and transdiagnostic realities experienced in the context of regular practice. This difficulty limits the ecological validity of standard EBI protocols, which are often developed to address one diagnosis or problem domain (Lyon, Dopp, Brewer, Kientz, & Munson, 2020). Adding to this complexity, there is a large degree of heterogeneity in frontline services in terms of professional level and clinical expertise as well as the resources that are committed to client contact (Tommeraas & Ogden, 2015). Accordingly, EBI protocols should be designed in light of this heterogeneity among clients and service systems as well as the need for flexible, efficient and usable interventions. The lack of ecological validity in EBIs may have contributed to the belief that these interventions are challenging to implement and sustain in regular practice settings and thus contributed to the gap between science and practitice as well as the limited scope of EBIs (Lyon et al., 2020).

A limited methodological scope in the context of evaluation may represent another challenge for innovation in the context of EBIs. The predominant methodology in EBI and evaluation science has been to evaluate EBI program packages using randomized controlled trials (RCT; Collins, 2018; Lyon et al., 2020; Schindler, McCoy, Fisher, & Shonkoff, 2019). Program package evaluation in RCT designs via the display of program package group mean effects has been vital for the progress of the EBI field. However, RCTs mask within-group heterogeneity in terms of intervention effects and active intervention ingredients, a difficulty which is often referred to as the black box problem in the context of EBIs (Kazdin, 2018; Lyon & Koerner, 2016; Schindler et al., 2019). The race to attain a high degree of internal validity in the context of expensive and time-consuming RCTs must be complemented with a research focus that allows us to look inside this black box and identify treatment nonresponders and the components that are effective for individual users; all of this information is necessary for innovation in the context of EBIs (Supplee & Duggan, 2019). The black box problem and the limited scope of EBIs warrant a shift in focus to the evaluation methodology that is useful for the design process alongside the more traditional and sorely needed evaluation of treatment packages in RCTs. The inclusion of various design-based methodologies to produce knowledge concerning the active ingredients of interventions may accelerate innovation and ultimately answer the following question: What works for whom in which contexts? (Collins, 2018; Schindler et al., 2017; Supplee & Duggan, 2019).

Design-Based Methodology

The need for intervention innovation is reflected in the necessity of additional design-based methodological elements that can be useful for the innovation process. First, a clear and precise specification of a Theory of Change (ToC; also known as a causal model or conceptual model) is a prerequisite and a tool for a rigorous research and development (R&D) process. A ToC refers to “a detailed set of beliefs about observable changes that are expected from an intervention” (p.147; Schindler et al., 2019). A well-specified ToC is vital to improve precision in the development and testing of an intervention (Collins, 2018; Supplee & Duggan, 2019). It is a tool for understanding what components and procedures work, why they work, for whom they work, under which circumstances they work, and on what outcomes they work. A well-specified and conceptualized ToC may serve as a tool for operationalizing the findings of basic and intervention research to advance the research-based development of EBIs. Moreover, a clear and conceptualized ToC is vital to the establishment of common ground in a design process featuring cocreation (Schindler et al., 2019).

Second, incorporating knowledge drawn from relevant stakeholders into the R&D intervention is pivotal for an effective design process (Klev & Levin, 2016). Hence, cocreation featuring relevant stakeholders (i.e., client families, service counselors and leaders) as well as program developers and researchers, both with respect to the research and the development of the intervention, will likely influence the usability and ecological validity of the EBIs (Lyon et al., 2020), allowing EBIs to be designed to fit stakeholder needs, knowledge, and service constraints. Ultimately, the cocreation of interventions will likely impact the scalability and sustainability of EBIs (Lyon & Bruns, 2019; Mummah, Robinson, King, Gardner, & Sutton, 2016).

Third, compared to traditional RCT designs, alternative evaluation designs are more appropriate to the task of addressing the active components and change mechanisms involved in the intervention. For example, the alternatives include sequential multiple assignment randomized trials, factorial experiments, single case experiments, or mixed methods designs (Collins, 2018; Kazdin, 2018; Palinkas, Horwitz, Chamberlain, Hurlburt, & Landsverk, 2011; Supplee & Duggan, 2019). Particularly, the latter two designs are well suited to rapid-cycle and iterative testing, in which context research hypotheses can be subsequently raised and tested, with the aim of moving rapidly toward higher levels of evidence, which is essential during the early stages of a design process (Gallo, Comer, & Barlow, 2013; Supplee & Duggan, 2019).

Innovative design-based frameworks that can accelerate EBI progress are emerging. These frameworks often share an introspective focus on active components as tools for innovation, and they propose different methodological elements that are important to the design process (Collins, 2018; Lyon et al., 2020; Schindler et al., 2017). One such model for the science-based development of interventions is the Frontiers of Innovations’ IDEAS Impact Framework (Innovate Evaluate Adapt Scale; Shonkoff et al., 2016). IDEAS highlights several methodological elements, such as cocreation in the context of intervention research and development to reflect stakeholders’ needs; precision in ToC specification and assessment to understand what works for whom and how; rapid-cycle iterative testing and optimization to accelerate our ability to gather evidence; and the establishment of an evaluation plan combining research, theory, and practice via the optimization of a ToC and the intervention materials (Schindler et al., 2017).

The Current Project: Aims and Research Questions

Incorporating a design-based methodology mainly based on the IDEAS impact framework but also on the multiple optimization strategy (Collins, 2018) and a user-centered design (Lyon & Koerner, 2016), the goals of this project are to (1) broaden the scope of the BPT-based intervention to reach a potentially large group of children and families in need of targeted (selected and indicated) preventive frontline services; (2) to design and optimize a ToC based on the BPT/PMTO and new knowledge drawn from relevant basic and intervention research; and (3) to cocreate a flexible and usable intervention that is tailored to stakeholders’ knowledge and needs. Thus, the R&D pertaining to the new intervention SPARCK is based on the constraints for the intervention’s development, including societal needs for mental health intervention, service constraints such as resources and personnel, and client needs for individualized and tailored intervention, with the aim of creating a scalable and sustainable intervention for routine practice.

We will test individual effects using single case experimental designs (SCEDs), systematic stakeholder feedback collected via qualitative interviews, and biomarker observations referencing parent and child cortisol stress hormones collected from hair samples. With the participation of service counselors and clients, we will cocreate a SPARCK prototype that combines BPT and new knowledge drawn from basic and intervention research concerning comorbid mental health symptoms and active ingredients for the target group, respectively. The aim of this effort is to design a unified and transdiagnostic preventive intervention with a broad theoretical and empirical foundation to address the heterogeneity in the challenges encountered in regular practice settings. All aspects of the SPARCK prototype, such as the intervention components, sequencing, dosage and program materials, will be optimized throughout the development and research phases, with the aim of achieving higher levels of efficacy and usability in the complete version of SPARCK. In addition, we will explore and cocreate implementation strategies that may further promote SPARCK’s scalability and sustainability in frontline settings.

The following research questions serve as a starting point for the R&D process.

• What families/children seem to benefit or not benefit from the SPARCK intervention?

• What seem to be the active components and change mechanisms for individual families?

• Is the ToC and the accompanying program material included in SPARCK usable and feasible for counselors working in Norwegian frontline services?

• What is the appropriate structure and sequencing of procedures and components for different users of SPARCK (including dosage, fixed component order vs. customized adaptation, decision support, etc.)?

• What implementation strategies are feasible and appropriate for supporting the sustainable implementation of SPARCK in the context of Norwegian frontline services?

The project has the potential to contribute new knowledge to the mental health prevention field in general, with respect to the benefit of utilizing innovative methodology in the intervention design process, as well as evidence concerning both the change mechanisms underlying a unified parenting intervention to promote healthy child development and the selection and tailoring of implementation strategies.

Developing SPARCK: Prototype Theory of Change

There is strong evidence to suggest that parent–child interaction is a mediator that is associated with long-term outcomes for children, including maltreatment (Green et al., 2018), internalizing problems (Shimshoni, Shrinivasa, Cherian, & Lebowitz, 2019), and externalizing problems (Nærde, Janson & Stoolmiller, in review). As a result, a parenting intervention in which parents are the agents of change is of particular relevance to a unified preventive intervention during childhood. In the initial design of the prototype ToC in SPARCK, we will follow the following tenets: 1) integrate new and relevant evidence drawn from basic research; 2) develop the model based on different theoretical intervention models to promote flexible ways of understanding and tailoring interventions to family needs; 3) incorporate new evidence from intervention research and identify the active components contained in effective interventions; 4) search for transdiagnostic approaches and components that may reduce the complexity of interventions and broadly promote healthy child development.

The prototype ToC is displayed in Figure 2. To adapt and tailor the intervention content and to promote different perspectives on ways of addressing individual family needs, the ToC of SPARCK is to some extent a theoretical hybrid, and the intervention components are influenced by different causal theories.

• Download figure
• Open in new tab

Figure 2.

SPARCK theory of changes in conceptual prototype

First, the causal theory included in BPT/PMTO, i.e., social interaction learning theory (SIL; Patterson, 2016), and the BPT component strategies that target externalizing problems serve as a point of departure for SPARCK (Forgatch & Patterson, 2010; Solholm et al., 2013). SIL highlights the way in which externalizing problems develop in the context of coercive parent–child interactions and the reciprocal process of negative reinforcement between children’s maladaptive behaviors and parents’ withdrawal and suboptimal parenting behaviors (Patterson, 2016).

Second, insecure attachment, particularly disorganized attachment, has been broadly associated with negative short- and long-term outcomes for children, such as poor emotional and self-regulatory capacities, mental health problems, cognitive problems, and dysregulation of cortisol hormones (Bernard et al., 2012; Cyr, Euser, Bakermans-Kranenburg, & Van Ijzendoorn, 2010; Euser, Alink, Stoltenborgh, Bakermans-Kranenburg, & van IJzendoorn, 2015). Parenting capacities constitute a core aspect of secure attachment (Cyr & Alink, 2017), and the literature concerning sensitive and nurturing parenting practices has influenced the ToC of SPARCK.

Third, emotional dysregulation has been associated with disrupted parenting practices as well as internalizing and externalizing symptoms in children (Fraire & Ollendick, 2013; Morris, Criss, Silk, & Houltberg, 2017). SPARCK is influenced by emotion socialization theory, which emphasizes the ways in which parents themselves can become aware of and learn to regulate their own emotions (Gottman, Katz, & Hooven, 1996) as well as the ways in which they can help their children deal with their emotions via emotional coaching (Johnson, Hawes, Eisenberg, Kohlhoff, & Dudeney, 2017). Emotional socialization interventions have been found to prevent and reduce mental health problems in children (Ansar et al., 2022; Bølstad et al., 2021; Edrissi, Havighurst, Aghebati, Habibi, & Arani, 2019).

Fourth, it has been suggested that parents could also be a mediator of change in interventions targeting internalizing symptoms. Support for parent/family-based cognitive behavioral therapy (CBT) has been reported by systematic reviews focusing on the treatment and prevention of anxiety disorders in young children (Comer, Hong, Poznanski, Silva, & Wilson, 2019; Schwartz, Barican, Yung, Zheng, & Waddell, 2019). SPARCK is influenced by the behavioral components of CBT by helping parents with respect to exposure to fears and behavioral activation to promote coping and specific skills that are necessary in daily life.

Fifth, the concept of family accommodation is incorporated in the ToC of SPARCK. Family accommodation refers to change parents make to their behavior, such as facilitating avoidance, excessive support and modifying family routines, to help the child mitigate feelings of distress and anxiety, which can negatively reinforce children’s problems and contribute to the maintenance or worsening of symptoms (Lebowitz, Panza, & Bloch, 2016). Compelling findings have shown that parental accommodation plays a significant role in the maintenance of anxiety symptoms (Shimshoni et al., 2019).

Even though SPARCK is inspired by different causal theories and components, there is some degree of overlap among these influences. For instance, several theories highlight the need for sensitive and responsive interactions. Such components that are common across theories of parent counseling may represent common components that are assumed to be important across a variety of problems and diagnoses. Other components are specific to one theory and address the specific problems for which the theory was originally developed. Thus, SPARCK is constructed based on the idea of utilizing general common components in combination with specific components that are tailored to specific problems. Hence, SPARCK consists of parenting components that are considered to be important for supporting children and families who display a broad range of problems. The conceptual prototype ToC, which is displayed in Figure 2, serves as a foundation for operationalizing component strategies in the SPARCK manual and intervention materials as well as a guiding tool for research focusing on mechanisms of change or the lack thereof.

A critical consideration in this context is the sequencing of the components and target strategies. Ultimately, two possibilities must be considered in the context of SPARCK: fixed sequencing versus customized adaptive sequencing. Ease of use may suggest linear sequencing; in contrast, adaptive sequencing may have an impact on the complexity of delivery, but it also allows for more flexibility to individualize content. In the SPARCK prototype, we start with a mixed solution featuring a linear basic sequence and an adaptive sequence, according to which all families will receive components listed above the dotted line in Figure 2 before the customization of the components listed below the dotted line. This solution may balance the need for personalized and individual tailoring and dosage with ease of use in the context of frontline services.

Recruitment of Service Counselors and Clients

An important aspect in this project is the lab, which includes stakeholders, professional counselors, their leaders, and end user families recruited from frontline services. The professional counselors will include 14 parent counselors who are accustomed to working with EBIs; these counselors will be recruited from seven municipalities that are representative of all Norwegian health regions. The counselors are recruited based on the following criteria: (a) they are trained in PMTO, (b) they are motivated to participate in the project, (c) they have access to the target groups, (d) they can deliver cases to research as part of their regular practice, and (e), they represent the relevant frontline service sectors that provide interventions to children and families. These counselors will represent municipal frontline services, comprising CWS services, PPT school-psychology services, and health services, all of which deliver some sort of prevention measures and/or treatments to children, youths, and families who struggle with socioemotional problems and parent–child interactions. These 14 counselors will be closely involved in the iterative R&D process as described below. In the following, when we use the term lab, we refer to these counselors.

The inclusion of study participants and client families in the project will follow the formal and clinical screening procedures used by the lab counselors ‘ in their regular frontline services. In frontline services, no diagnosis is necessary to receive help, which reflects the selected and indicated level of prevention. In addition, formal screening procedures will be implemented prior to inclusion in the iterative test cycles (as presented in the assessment procedures). Client participants will typically be parents of children who display internalizing and/or externalizing symptoms and/or parents who are at risk of exhibiting maladaptive or neglectful parenting practices. Clients will be excluded if it is indicated that the target child has been exposed to physical violence or sexual abuse or if children and/or parents are receiving other ongoing treatments related to the target group domains of SPARCK.

Research and Development Design

Results – Optimization Decisions

The number of iterative cycles that should be conducted depends on the results obtained in each cycle. In cycle 1, we plan to test the ToC in the context of a limited age span and symptom domain, including parents and their children, with the latter being between the ages of 6 and 10 years and displaying signs of internalizing problems. The main reason for this approach is to reduce participant heterogeneity in the first cycle to ensure sufficient replications of relatively homogenous cases in terms of age and symptom domain. Moreover, we opt to test all components of ToC in cycle 1 to learn more about their applicability to and efficacy for families. Depending on the results obtained in cycle 1, subsequent cycles will include the remaining age spans and/or symptom domains to test the SPARCK ToC with respect to the complete target group. Moreover, depending on the results and feedback obtained from stakeholders, we may also modify the research design and add and/or remove measurement instruments.

A period of analysis, cocreation activities and optimization will follow each test cycle. The results obtained via the different measures will be combined and synthesized to determine what works or does not work for various subjects and to try to reveal systematic patterns of change or a lack of change both in and between cases. Similarly, we hypothesize that these qualitative data will contribute to improving our understanding of the change mechanisms associated with families in combination with the quantitative measures. In cases featuring discrepancies between measurement instruments and/or methods, qualitative evaluations will inform the optimization decisions. It is important to note that we will not use predefined criteria, such as effect size cutoffs, to determine optimization decisions. The decision concerning when to transition between phases in the innovation plan will be based on data integration and a determination of when the optimization changes are approaching a satisfactory level of detail, indicating that an acceptable level of confidence regarding who will benefit from SPARCK has been achieved.

Ethical considerations

The project has been reviewed and approved by the Regional Committees for Medical and Health Research Ethics, Southern and Eastern Norway (REK South East). The project involved certain ethical considerations concerning the families receiving SPARCK and their potential vulnerable life situation. However, SPARCK builds on well-established and tested components, indicating that its content is safe and robust. We will use the University of Oslo’s services for sensitive data (TSD) for data management, including collection, handling, analysis and storage procedures and follow standards of collection and storage of biological material. Participants will sign written informed consent forms prior to their inclusion in the project, and their anonymity and confidentiality will be secured in the project’s outputs.