Abstract
Background The CAPS is regarded as the “gold standard” in PTSD assessment. It is a structured interview that yields a categorical diagnosis of PTSD and also a measure of the severity of PTSD symptoms. Since a PTSD diagnosis in some settings could be connected to getting benefits, scoring inconsistencies may be more abundant with this rating scale as they not only reflect raters carelessness but intentional inaccurate reporting by the subject.
Objective The objective of the current effort was to derive rating consistency flags for the CAPS-IV.
Methods We deconstructed CAPS scoring instructions and anchors to identify potential scoring inconsistency flags. These inconsistency flags were reviewed and confirmed by expert raters. To test the ability of the flags to identify careless responses the flags were applied to Monte Carlo simulated data of 100,000 CAPS administrations.
Results Twelve flags were derived (presented in Table 1). Two flags applied to most of the 17 symptom items (Flag 1: Frequency=0 & Intensity>0 and Flag 2: Frequency>0 & Intensity=0). The remaining 10 flags pertained to individual items. Five flags were rated as “High” flags, representing very probably or definitely incorrect rating, one as medium, reflecting probably incorrect rating. Flags were raised for 95% of the Monte Carlo simulated CAPS administrations, 78% of the administrations had 4 or more flags and 60% 5 or more.
Conclusions Scoring consistency flags for the CAPS may be useful in the quest to improve reliability and validity of clinical trials. Modified flags are currently being developed to cover the CAPS-V. Further testing of flags using clinical trial data is planned to examine their potential impact on signal detection.
Introduction
The CAPS is regarded as the “gold standard” in PTSD assessment. It is a structured interview that yields a categorical diagnosis of PTSD and also a measure of the severity of PTSD symptoms. It can be administered in 30-60 minutes by a trained rater.
To improve the reliability and validity of measurement in clinical trials, we previously developed consistency checks “flags” for the Montgomery-Asberg Depression Rating Scale (MADRS)(Rabinowitz et al., 2019, 2022), Positive and Negative Syndrome Scale (PANSS) (Rabinowitz et al,, 2017; 2021), Personal and Social Performance scale (PSP) (Rabinowitz et al, 2021), the Young Mania Rating Scale (YMRS) (Rabinowitz et al, 2024).
Since a PTSD diagnosis in some settings could be connected to getting benefits, scoring inconsistencies may be more abundant with this rating scale as they not only reflect raters carelessness but intentional inaccurate reporting by the subject.
Aim
The objective of the current effort was to derive consistency flags for the CAPS-IV.
Methods
The first 17 items of CAPS IV elicits ratings on Frequency (0=never; 1=once or twice; 2=once or twice a week; 3=several times a week; 4-daily or almost every day) and intensity (0=no distress; 1=mild; 2=moderate; 3=severe; 4=extreme) of symptoms and are used to compute a total severity score. The next 4 items measure duration, subject distress and functional impairment. A scoring algorithm is applied to these 21 items to arrive at a diagnosis. The CAPS also includes 3 global ratings and ratings of 5 associated features.
We deconstructed CAPS scoring instructions and anchors to identify potential scoring inconsistency flags. These inconsistency flags were reviewed and confirmed by expert raters.
To test the ability of the flags to identify careless responses the flags were applied to Monte Carlo simulated data of 100,000 CAPS administrations.
Results
Twelve flags were derived (presented in Table 1). Two flags applied to most of the 17 symptom items (Flag 1: Frequency=0 & Intensity>0 and Flag 2: Frequency>0 & Intensity=0). The remaining 10 flags pertained to individual items. Five flags were rated as “High” flags, representing very probably or definitely incorrect rating, one as medium, reflecting probably incorrect rating. Flags were raised for 95% of the Monte Carlo simulated CAPS administrations, 78% of the administrations had 4 or more flags and 60% 5 or more. Two high flags, Flag 1 and 2 were raised in more than 85% of the administrations. Table 2 presents flags for both the CAPS-IV and CAPS-V and a comparison of the items on each scale.
Conclusions
Scoring consistency flags for the CAPS may be useful in the quest to improve reliability and validity of clinical trials. Modified flags are currently being developed to cover the CAPS-V. Further testing of flags using clinical trial data is planned to examine their potential impact on signal detection.
Data Availability
All data produced in the present work are contained in the manuscript.
Acknowledgments
The research leading to these results has received support from the Elie Wiesel Chair at Bar Ilan University.
Previous versions of this material have been presented at ISCTM 2021 Autumn Conference
Jonathan Rabinowitz has received research grant(s) support and/or travel support and/or speaker fees and/or consultant fees from Janssen (J&J), Eli Lilly, Pfizer, Lundbeck, Roche, Abraham Pharmaceuticals, Pierre Fabre, Intra-cellular Therapies, Minerva, Takeda and Amgen. Alon A. Rabinowitz and Sara Freedman have no disclosures.
