Abstract
Background Disease severity is considered to be an important factor for inclusion of diseases on expanded carrier screening (ECS) panels. Evaluating severity requires an objective, systematic method. Here, we applied a validated algorithm that objectively categorizes disease severity into one of four categories—profound, severe, moderate, and mild—to 176 genes on a clinically available ECS panel.
Methods Eight genetic counselors (GCs) working in pairs, followed by four medical geneticists (MDs) working in pairs, applied the algorithm to subsets of the 176 genes.
Results Upon initial GC and MD review, 107/176 genes (61%) and 133/176 genes (76%), respectively, had concordant classifications, with consensus reached for all genes after collaborative review. Final severity classifications were 68 (39%) profound, 71 (40%) severe, 36 (20%) moderate, and one (1%) mild. No classification biases were detected within the GC or MD pairs.
Conclusion This study illustrates an approach to severity classification for large sets of Mendelian genes. The observed level of initial discordance demonstrates the complexity of severity and underscores the importance of collaboration involving multiple clinicians. Severity classification, in addition to other factors suggested by ACOG, is an important factor for laboratories to consider as they aim to design clinically valid ECS panels.
Introduction
Carrier screening seeks to identify couples at risk of conceiving a pregnancy affected by a genetic disease, thereby informing reproductive decision-making and pregnancy management. Expanded carrier screening (ECS), which has become increasingly utilized, accomplishes this for dozens to hundreds of diseases, in comparison to traditional ethnicity based screening approaches meant to detect limited conditions prevalent within those ethnic groups (Haque et al., 2016). The American College of Obstetricians and Gynecologists (ACOG) considers ECS to be an acceptable screening strategy and has proposed multiple criteria for disease inclusion on ECS panels (ACOG Committee on Genetics, 2017). One criterion addresses disease severity, suggesting that disorders selected for screening panels should have a detrimental effect on quality of life, cause cognitive or physical impairment, require surgical or medical intervention, or have an onset early in life (ACOG Committee on Genetics, 2017). Similarly, the American College of Medical Genetics and Genomics (ACMG) position statement on ECS states that “disorders should be of a nature that most at-risk patients and their partners identified in the screening program would consider having prenatal diagnosis to facilitate making decisions around reproduction” and that “there must be a validated clinical association between the mutation(s) detected and the severity of the disorder” (Grody et al., 2013). However, the concept of severity can be subjective; individuals have varying perceptions of it based on their valuation of phenotypic characteristics (Kraft et al., 2018; Lazarin et al., 2014).
Disease severity inclusion criteria for ECS panels have not been widely studied. However, multiple studies have shown that disease categorization is helpful for patients in understanding the types of diseases included on ECS panels and facilitates reproductive decision-making (Ghiossi, Goldberg, Haque, Lazarin, & Wong, 2018; Johansen Taber et al., 2019; Kraft et al., 2018).
In 2014, Lazarin et al. published a framework to objectively categorize disease severity. In the study, 192 health care providers rated 13 individual disease traits, independent of any named genetic disease. Disease traits (e.g., limited life span, intellectual disability), as opposed to diseases themselves, were evaluated because, despite health care providers’ potential lack of familiarity with many rare genetic diseases, the traits associated with them are often encountered by health care providers regardless of the etiology. The analysis established and validated an algorithm that can be applied to a disease, with its given set of individual phenotypic traits, resulting in a classification of the condition into one of four severity categories: profound, severe, moderate, and mild (Lazarin et al., 2014). The phenotypic characteristics that define each severity category correspond to the criteria ACOG has set forth for determining whether the severity of a given disease warrants inclusion on an ECS panel (Table 1). Both the profound and severe categories are characterized by a detrimental effect on quality of life, cognitive or physical impairment, a requirement for surgical or medical intervention, and an onset early in life. The moderate category is characterized by a detrimental effect on quality of life, a requirement for surgical or medical intervention, and an onset early in life. The mild category does not meet any of the phenotypic characteristics set forth by ACOG.
Severity categories as defined by Lazarin et al. (2014) and their alignment with ACOG criteria for inclusion in ECS panels (ACOG Committee Opinion 690).
The health care provider participants in Lazarin et al. (2014) were comprised largely of physicians and genetic counselors working in reproductive or obstetrics settings and did not include participants with specialized knowledge of the diseases used in the validation of the algorithm. To date, no published studies have rigorously applied the algorithm using health care providers with such specialized knowledge, nor has it been applied to the large number of diseases currently included on many ECS panels. In this study, we aimed to assign an objective severity classification to genetic diseases in order to inform consideration of their inclusion on ECS panels. We addressed the limitations of the previously described study by engaging genetic counselors from clinical pediatric settings and medical geneticists who have expertise in rare diseases commonly found on ECS panels. These health care providers applied the algorithm to the 176 genes found on a commercially available ECS panel to classify each into severity categories.
Materials and Methods
Editorial Policies and Ethical Considerations
This study did not use patient samples or results and therefore was not subject to review by an institutional review board or ethics committee.
Gene Classifications
Eight genetic counselors (GCs) from pediatric clinical settings and four medical geneticists applied a published severity algorithm to randomly assigned subsets of 176 genes offered on a commercially available ECS panel (Foresight, Myriad Women’s Health) (Lazarin et al., 2014). The algorithm was applied in a two-step review and consensus process as described below. Honoraria were provided to each GC and medical geneticist participant.
In the first step, the 176 genes were randomly divided into four subsets of 44, and a pair of GCs was assigned to each subset so that each gene was evaluated by at least two GCs (Figure 2). Each GC used the algorithm (Figure 1) to independently classify the assigned genes and the associated phenotype arising in untreated, affected individuals (i.e., those with one or two pathogenic mutations, depending on the inheritance pattern of the condition) as profound, severe, moderate, or mild. After the initial independent review, each pair of GCs was notified of genes classified discordantly between them and were then asked to review their initial severity classifications together to come to a final classification consensus for each gene. Factors contributing to initial discordant classifications were collected and reviewed.
Decision framework for severity classification algorithm published by Lazarin et al. (2014) and utilized in this study. Each GC and MD used the framework to independently review and classify genes and the associated phenotype arising in at least 25% of individuals in the untreated state, as profound, severe, moderate, or mild.
Workflow of the severity classification process. Eight genetic counselors (GCs) were divided into four pairs; each pair was asked to review 44 genes. Once the GC pairs finished their initial review and resolved discordances, the classification process was then passed to the four medical geneticists (MDs) for review and final classification. Gene numbers in this figure correspond to the gene numbers in Supplemental Table 1.
In the second step (following GC review), two pairs of medical geneticists (referred to throughout as “MDs”) used the algorithm (Figure 1) to independently classify the assigned genes and the associated phenotype, with each MD pair reviewing 88 genes (Figure 2). Final GC severity classifications were available to the MDs prior to their individual evaluation. After independent review, each MD pair was notified of genes classified discordantly within the MD pair and/or discordantly between MD and GC final classifications. The MD pairs were asked to reach a final consensus classification and provide factors that contributed to this final classification decision. Factors contributing to discordances between MDs, as well as factors contributing to the final classification, were collected and reviewed.
Data Analysis
Descriptive statistics were used to characterize general data trends. Statistical significance between proportions was determined using chi-square analysis; a result was considered significant when p□<□0.05 at the 99% confidence level.
Permutation Testing
Permutation testing was conducted to determine whether particular individuals within each GC (1-4) or MD (1-2) pair consistently classified disease genes as higher or lower in severity than their counterpart. Qualitative disease severity classifications were converted to a numeric scale ranging from 1 to 3, with 1 being the least severe classification (mild) and 3 being the most severe classification (genes classified as severe or profound were collapsed into one category as they both meet the same ACOG inclusion criteria [Table 1]). For genes assigned to MD Pair 1 where one individual failed to classify a gene, the gene was ignored. Using the numerical scale of severity, we then took the mean of the gene set assigned to each pair for each individual and measured the absolute difference between the two GCs or two MDs to generate an observed value of difference.
To run the permutation test, observed classifications within each GC pair and MD pair were permuted for each gene by randomly switching the observed classifications given by the individuals within the pair, repeating 100,000 times for the full set of assigned genes for each pair. After each permutation iteration, the mean of severity classifications was measured across the assigned genes and the absolute difference was calculated among the two members in each pair, generating a permuted value of difference. Significance was assessed by calculating the fraction of permuted values of difference that were greater than or equal to the observed value for a particular GC or MD pair, generating an empirical p-value for the permutation test.
Results
Genetic Counselor Review and Classification
After initial review by the GC pairs, 107 of the 176 disease-associated genes (61%) had concordant classifications (Figure 3A, Supplemental Table 1). Concordances between the GC pairs ranged from 47.7% to 68.2%, with 30 concordant classifications (68.2%) after initial review by GC Pair 1, 21 (47.7%) after initial review by GC Pair 2, 29 (65.9%) after initial review by GC Pair 3, and 27 (61.4%) after initial review by GC Pair 4. With the exception of four genes (NR0B1, ABCC8, KCNJ11, CYP21A2), all discordant classifications were within one level of severity classification (Figure 3A, Supplemental Table 1). A permutation test was conducted to determine whether any individual within each GC pair tended to classify genes as more or less severe compared to their counterpart. For the permutation test, genes classified as severe or profound were collapsed into one category as they both meet the same ACOG inclusion criteria (Table 1). We found no significant difference in the tendency of any individual within a GC pair to classify disease genes as more or less severe on average than his/her counterpart (all p > 0.05; Figure 4).
Concordance after initial gene severity classification. Concordance of initial severity classifications within each GC pair and in total among all GCs (A), and concordance of initial severity classification within each MD pair and in total among all MDs (B). Levels of discordance are indicated by color (pink: concordant, green: discordant by one level, blue: discordant by two levels, purple: not classified).
Permutation testing to detect classification bias. Mean differences were calculated within each GC and MD pair after permuting observed classifications 100,000 times. Classifications were converted into a numerical scale, with ‘Severe’ and ‘Profound’ severity classifications collapsed (Mild=1, Moderate=2, Severe=3, Profound=3), and the mean difference of all disease classifications were calculated within each pair for each round of permutation. P-values represent the proportion of permutations that were greater than or equal to the observed classification differences within each GC and MD pair.
Reasons for discordances noted by the GC pairs included difficulties in determining the primary phenotype associated with a specific disease with varying severity, difficulty discerning the level of intellectual disability associated with the disease from the available literature, and inability to locate published data regarding life expectancies and phenotypic differences in various forms of the conditions. After final review of discordances and consensus on final classifications between each GC pair, 65 genes (36.9%) were categorized as profound, 65 (36.9%) as severe, 42 (23.9%) as moderate, and four (2.3%) as mild (Supplemental Table 1).
Medical Geneticist Review and Classification
After initial review by each member of the MD pair, 133 of the 176 disease-associated genes (76%) had concordant classifications (Figure 3B, Supplemental Table 1). Concordance within the two MD pairs ranged from 67% to 84.1%, with 74 (84.1%) concordant classifications after initial review by MD Pair 1 and 59 (67%) after initial review by MD Pair 2. One member of MD Pair 1 did not definitively classify five genes; these were considered as discordances. All but one discordant classification (SLC22A5) was within one level of severity classification (Figure 3B, Supplemental Table 1). A permutation test was conducted to determine whether any individual within each MD pair tended to classify genes as more or less severe compared to his/her counterpart. As noted above for analysis of GC discordances, genes classified as severe or profound were also collapsed into one category for permutation testing as both categories meet the same ACOG inclusion criteria (Table 1). No significant difference was observed in the tendency of any individual within an MD pair to classify disease genes as more or less severe on average than his/her counterpart (MD Pair 1: p=0.218, MD Pair 2: p=0.421; Figure 4).
Reasons for discordances included no definitive classification, multiple phenotypes associated with the gene, unknown percentages of individuals with intellectual disability or reduced lifespan, and difficulties in determining life expectancy in the untreated state for conditions for which treatment is available. After final MD review of discordances and consensus on final classifications, 68 (38.6%) genes were categorized as profound, 71 (40.3%) as severe, 36 (20.5%) as moderate, and one (0.6%) as mild (Supplemental Table 1). Comparison of GC and MD classifications revealed no significant differences in the number of genes ultimately classified within each severity category (36.9% vs. 38.6% Profound, p=0.74; 39.6% vs. 40.3% Severe, p=0.51; 23.9% vs. 20.5% Moderate, p=0.44; 2.3% vs. 0.6% Mild, p=0.18).
Discussion
The severity classification framework applied here provides a systematic approach to evaluate the severity of genes that are included in ECS panels. Herein, health care providers with expertise in pediatric and genetic disease used it to classify the severity of 176 genes on a commercially available ECS panel. Professional consensus was reached in the event of initial classification discordance, however, discordances among health professionals with specialized expertise demonstrate the potentially subjective nature of severity.
Subjectivity in Guidelines
Guidelines stipulate severity as an important criterion for ECS panel inclusion yet describe the criterion in terms that can be subjectively interpreted. For example, ACOG states that “disorders should have a detrimental effect on quality of life, cause cognitive or physical impairment, require surgical or medical intervention, or have an onset early in life.” But “detrimental” could apply to a range of phenotypes, and “onset early in life” could mean any time before adulthood. Impairments and interventions can be binarily defined—e.g., both cardiac surgery and hearing aids are interventions—but it is unclear as to what type of intervention is sufficient to qualify a disorder for inclusion. Many rare diseases screened by ECS do not have treatments that reverse the course of disease, but nevertheless have interventions that improve the quality of life for those affected by delaying or halting the onset of symptoms or lessening the severity of one or more phenotypic characteristics of the disease.
ACMG states that disorders included on ECS panels “should be of a nature that most at-risk patients and their partners identified in the screening program would consider having a prenatal diagnosis to facilitate decision making around reproduction.” We agree with the sentiment that providers and couples fully discuss the disease for which the couple’s current or future pregnancy is at risk, including the severity of the disease, to determine the couple’s desire to undergo prenatal diagnosis or interventions to reduce the risk of an affected pregnancy. But the ACMG guidance provides little direction for panel design. It implies that patients themselves determine the disorders to be included but does not address how this could logistically occur. The majority of patients are unfamiliar with conditions on carrier panels (as are many providers), so a large-scale study of patients’ willingness to undergo diagnostic testing for each condition may not be feasible. Indeed, this is the reason that Kraft et al. (2018) grouped conditions into severity categories when asking patients the diseases for which they preferred to be screened. The approach suggested by ACMG also conflicts with the norm of practice guidelines originating from a review of evidence by medical professional societies rather than community views (ACOG Committee on Genetics, 2017; Grody et al., 2013).
Both the ACOG and ACMG statements aim to provide simplified guidance regarding severity in the face of complex variability in genetic disease. Previous studies have attempted to specifically define the range of phenotypes caused by a genetic disease (Korngiebel et al., 2016; Lazarin et al., 2014), but our study demonstrates the complexity of the notion of severity. Initial classification discordance by health care providers with expertise in pediatric and genetic disease demonstrates that severity is subjective and may not fit neatly into categories. The fact that both profound and severe conditions meet the same ACOG severity criteria is yet another indicator of the difficulty in definitive classification. These data suggest that severity cannot be strictly defined and that rigid severity level requirements for inclusion on ECS panels should be reconsidered.
Utility of screening for moderate severity conditions by expanded carrier screening
The term “moderate” as a descriptor of severity may connote to some that the disease does not rise to a level of significance for a couple wanting to know their risk for having a pregnancy affected by genetic disease. However, moderate severity conditions meet three of the four criteria set forth by ACOG as important for choosing conditions to be included on an ECS panel: they have a detrimental effect on quality of life, a requirement for surgical or medical intervention, and an onset early in life (ACOG Committee on Genetics, 2017). As an example, GJB2, variants in which cause non-syndromic hearing loss and deafness, was categorized by this study as having moderate severity. This condition has an onset at birth, and numerous studies have demonstrated the importance of early identification and intervention for individuals with hearing loss. If not detected early, hearing loss complicates childhood development in language, socialization, academic performance, and most importantly human development and self-actualization (Fulcher, Purcell, Baker, & Munro, 2012; May-Mederake, 2012; Moeller, 2000; Poonual, Navacharoen, Kangsanarak, & Namwongprom, 2017). In addition, interventions such as cochlear implants can effectively avert these developmental delays (Fulcher et al., 2012).
Previous studies have also shown that couples find value in screening for moderate severity conditions. Two clinical utility studies found that a majority of couples identified by ECS to be at risk for pregnancies affected by moderate severity conditions made reproductive and pregnancy management decisions for future family planning based on knowledge of such risk (Ghiossi et al., 2018; Johansen Taber et al., 2019). A separate study assessing patient preferences for receipt of carrier screening results found that patients preferred to receive ECS results on conditions in all severity categories, including moderate, and reported that severity classifications informed their choice as to the conditions for which they wanted to undergo screening (Kraft et al., 2018). These studies imply that couples value and utilize ECS results for conditions across the severity spectrum. The utility of early identification and intervention for genetic conditions, even those categorized as moderate severity, along with patient preferences for screening for moderate severity conditions, suggests that they should be included in carrier screening panels.
Categorizing genes vs diseases
Next-generation sequencing advances have led to increasingly comprehensive ECS assays, both in number of diseases tested and in mutational coverage of genes. While greater mutation detection increases clinical sensitivity, it also reveals the phenotypic spectrum caused by varying mutations within a given gene. Indeed, this variable expressivity was a cause of discordant classifications upon initial review by both GCs and MDs, though they were ultimately able to reach a consensus classification. We propose that, rather than considering diseases to be included on ECS panels, a discussion should instead focus on the specific genes, the approach taken by this study. This was necessary because some conditions are caused by multiple genes (i.e. Usher Syndrome, Fanconi Anemia, Niemann Pick Disease) and some genes are associated with multiple conditions (i.e. HBB, FKTN, MYO7A). Additionally, growing evidence illustrates the difficulty in correlating genotype and phenotype for certain genetic conditions with variable penetrance (Strande et al., 2017). The framework published by Lazarin et al. (2014) and applied here takes these complexities into consideration and focuses on characteristics present in at least 25% of individuals with mutations in a given gene.
Complexity of ECS panel design
ECS is complex by nature, typically involving dozens of conditions; addressing these complexities as they arise helps to optimize patient care. Both this study and Kraft et al. (2018) revealed the difficulty in assigning severity to rare conditions, often because published literature about such conditions is sparse, phenotypes are variable, and availability of treatment for certain conditions is limited. Yet difficulty in assigning severity does not diminish the need to provide counseling and education to patients about the severity of conditions that they or their family members are facing. Several analyses have suggested additional factors to be considered in optimal panel design, including clinical sensitivity, actionability, and prevalence of each condition (Beauchamp et al., 2018; Ben-Shachar, Svenson, Goldberg, & Muzzey, 2019; Chokoshvili, Vears, & Borry, 2018; Stevens, Krstic, Jones, Murphy, & Hoskovec, 2017). ACOG also suggests several criteria other than severity that should guide ECS panel design, including a carrier frequency of one percent and the ability to diagnose the disease prenatally (ACOG Committee on Genetics, 2017). We suggest that conditions categorized as moderate severity or higher meet ACOG’s severity criteria for inclusion on a panel, and that ACOG’s and other researchers’ additional suggested factors be used in combination to optimize panel design. For example, a panel consisting of diseases that meet at least three of the four ACOG severity criteria (those in the profound, severe, and moderate categories, Table 1), have a carrier frequency of at least one percent in any ethnicity (Ben-Shachar et al., 2019), and that can be diagnosed prenatally would be made up of approximately 40 diseases, indicated by daggers in Supplementary Table 1.
Limitations
This study has limitations that should be noted. First, the GC and MD participants were selected to review the conditions because of their expertise in pediatrics and medical genetics, but their application of the severity algorithm may not replicate application by other clinicians with similar or different expertise. Second, the GC and MD participants may themselves have had biases in categorization toward more or less severity. We attempted to account for potential biases using permutation testing, and found no significant differences in categorization among the pairs. Third, the MDs were not blinded to the final GC classifications, so it is possible that their classification was influenced by that of the GC pair. However, this paradigm emulates clinical practice, in which GCs (or other providers) routinely present initial findings to their provider colleagues as the entire care team manages the patient. Fourth, genes were assessed based on the available literature, which may skew toward more severe disease presentations, particularly in the case of rare diseases. As understanding of phenotypes evolves, the classifications may change. And finally, the framework used in this study to categorize severity may itself have had limitations (Lazarin et al., 2014).
Conclusions
This study applied a systematic and transparent process to evaluate the severity of genes on a commercially-available ECS panel by engaging with a diverse set of genetics providers with expertise in rare disease. All but one gene were classified into profound, severe, or moderate severity categories, all of which meet at least three of the four ACOG criteria for severity of conditions to be included on an ECS panel. Severity classification, in addition to other factors suggested by ACOG and community consensus, can be used by laboratories to consider genes for inclusion on expanded carrier screening panels.
Data Availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on request.
Data Accessibility
The data that support the findings of this study have been completely reported in this manuscript and shared in the Figures and Supplementary Material.
Disclosure Statement
Aishwarya Arjunan, Holly Bellerose, Katherine Johansen Taber, Raul Torres, Jennifer Tarpinian, and Gabriel A. Lazarin are all current or former employees of Myriad Women’s Health.
Acknowledgements
The authors thank Danielle Fanslow for design and editorial assistance. This study was funded by Myriad Women’s Health.
Disclosure: AA, HB, KJT, JT and GAL are all current or former employees of Myriad Women’s Health.
Footnotes
↵‡ JT was an employee of Children’s Hospital of Philadelphia at the time of study and has since moved to Myriad Women’s Health. GAL was an employee of Myriad Women’s Health at the time of study.
