Developing and testing an arts-based, digital knowledge translation tool for parents about childhood croup

Iterative Development Process

Active parent and stakeholder engagement was a key focus of all stages of the iterative development process of the whiteboard animation video. A timeline of the processes involved is included in Appendix A. Previous research by the study team collected and analyzed qualitative data on parental experiences with childhood croup to inform the development of a composite narrative for the whiteboard animation video, complemented by research-based recommendations from the TRanslating Emergency Knowledge for Kids (TREKK) Bottom Line Recommendations (BLR) [16, 18]. The interview guide used to elicit parents’ experiences can be found in Appendix B. Results from these interviews are published elsewhere [16]. Updates to the research-based recommendations are completed regularly, with literature searches updated on a bi-annual basis [18]. Through a competitive process, we selected an animation studio with writers, digital animators, and voice actors to develop a storyboard, script, and prototype video. The prototype was shared with two groups of parents with national representation: 1) the TREKK Parent Advisory Group, 2) the Canadian Family Advisory Network. The same prototype was also shared with emergency medicine clinicians at the annual Pediatric Emergency Research Canada (PERC) research meeting. Feedback was solicited from these stakeholders and used to refine the video content, script, and visuals.

The video was 2 minutes and 56 seconds, narrated in the third person, and portrayed a single mother and her young male child’s experience with croup. The story is set around bedtime when the child suddenly starts exhibiting a cough. The video includes a sound clip of a croup cough, which is often distinct from other types of cough. The narrator describes what croup is and provides measures parents could take to manage a mild case of croup. The video also provided parents with information about what is likely to happen if they were to go to the hospital for croup symptoms. Finally, the video also includes information about preventative measures parents could take against croup, including washing hands and cleaning surfaces (Appendix C).

Usability refers to the capacity of a tool or intervention to be used easily, efficiently, and satisfactorily by people [19]. Offering interventions or tools in a digital format enables more people to benefit from the intervention, however ensuring the digital intervention is usable is critical prior to widespread dissemination [20-22]. Therefore, this study employed a multi-method approach.

Surveys

Parents were recruited to participate in an electronic survey in three ED waiting rooms across Canada representing urban (Stollery Children’s Hospital), rural (Portage District General Hospital), and remote (Stanton Hospital) health regions. Members of the study team approached parents in the ED to determine interest and study eligibility. Study team members were available in the waiting rooms to provide technical assistance and answer questions as parents were completing the surveys.

Survey content was informed by a systematic search of over 180 usability evaluations [19]. The survey was comprised of 8, 5-point Likert items assessing: 1) usefulness, 2) simplicity, 3) level of engagement, 4) satisfaction, 5) quality of information, and 6) perceived value (Appendix D). Two free text boxes were also included.

Data were collected using the FluidSurveys platform on iPads. This platform allows for synchronous and asynchronous data collection and data is stored on a secure, Canadian server [23]. As previously reported, iPads were optimized with rigorous security features, including passcode login, data encryption, GPS tracking, and remote wipe capability [24]. Survey data was cleaned and analysed using SPSS v. 24 [25]. Descriptive statistics and measures of central tendency were generated. Survey data from the free text boxes was analyzed using content analysis.

Focus Groups

Parents were purposively recruited to participate in one focus group in each of the three regions. Focus groups are an efficient, cost-effective data collection method that provides opportunities to generate rich data while also observing group dynamics, and levels of consensus on topics [26-27]. In two regions, parents were recruited from existing parent advisory groups involved in child health research. In the third region, the local public health unit distributed flyers about the focus group to their parent groups.

The semi-structured focus group guide was comprised of questions about: a) video viewing experience, b) video attributes that were useful, c) video elements that were not helpful, d) perceptions of the video’s utility, and e) recommended revisions (Appendix E). Each aspect of the video (i.e., narrative, visual appeal, health information, engagement and interactivity) was explored. Focus groups were audio recorded on iPads with the same security features. Digital recordings were securely transferred to a third party for verbatim transcription. Focus group transcripts were cleaned and analyzed using NVivo 11 [28]. Data was analyzed deductively using the semi-structured interview guide to develop broad categories. Focus group data was then coded, codes were placed into the broad categories, and the categories synthesized into themes. Findings from the focus groups were used to inform final revision of the tool prototypes.

Survey Results

Thirty-eight surveys were completed by parents. Participant demographics are presented in Table 1. The results of the usability survey measures are presented in Table 2. Participants gave favourable scores for all 8 usability questions, especially for “it is simple to use” and “I am satisfied with it”. Further analyses were conducted to compare responses based on region (Urban vs. Rural vs. Remote and Urban vs. Non Urban). Results did not show significant differences between the sites.

The survey contained two free-text questions asking participants to list the most positive and negative aspects of the video. Thirty-four participants provided feedback on positive aspects of the video and their responses were grouped into four main categories: informative, easy to understand, short and to the point, and visual appeal (n=4). Fifteen participants provided feedback on negative aspects of the video, which were grouped into four main categories: narrator spoke too fast, too wordy, too long, and the moving hand drawing the content was distracting.

Focus Group Findings

Nineteen parents participated in 3 focus groups; one focus group in each of the three regions. The average number of participants was six per focus group. Participant demographics are presented in Table 3. Focus group length ranged from 23 minutes to 42 minutes.

Participants enjoyed the whiteboard animation medium. In particular, the parents liked that the video about croup was succinct, well-paced, and had a realistic and informative storyline. In addition to the helpful information provided, the video resonated with parents: “I really liked [the croup video] because it brought back my own memories and the stories and what that felt like. So it would give you an emotional response, which was great, because I can imagine at that time, how that would be a very good video to watch” (Participant 7, focus group 2).

While the short, succinct nature of the video was largely described as an asset, some parents did want more information on the cause of the illness, how to contact health providers, and how to prevent future bouts of croup: “What I liked about [the croup video] was how to avoid it in the future […] ‘Cause I think prevention’s just as important as… you know, what to do when it’s acute and you have to get to the Emerg.” (Participant 1, focus group 1). They also identified issues of representation, such as the importance of including diversity (i.e., race, gender, language, etc.), considering social-demographic factors in the video narrative: “The baby monitor is irrelevant, whether you hear you baby coughing through a baby monitor or not. And some families don’t have baby monitors. And driving to the hospital […]. That’s irrelevant. It’s - get them to the hospital.” (Participant 1, focus group 2). Participants suggested translating the video and adding sub-titles to reinforce the audio content and make the video accessible to those with hearing impairments: “What I’m seeing online is that they’re putting subtitles on – for everything, for people who are, you know, hard of hearing or for people who just don’t want their volume on – which if you have a sleeping baby, sometimes they don’t want the volume on, right?” (Participant 6, focus group 3). Parents also indicated that delivering this information at the appropriate time is an important consideration and the timing might not always support the use of a video: “Well, it depends how I’m gonna access the information and when I need it. So… you know, when somebody hands me something in the Emergency Room, I’m probably gonna refer to it later and ask the questions of the person who’s standing in front of me. But I like the videos very much for, if I have questions and I Google, “croup” and the video shows up, in less than three minutes I get some good information about, ‘don’t panic - this is what’s going on. Here’s how you determine if you need to go any further’” (Participant 1, focus group 1).