An Evaluation of Empirical Approaches for Defining Cognitive Impairment in Amyotrophic Lateral Sclerosis ======================================================================================================== * Corey T. McMillan * Joanne Wuu * Katya Rascovsky * Stephanie Cosentino * Murray Grossman * Lauren Elman * Colin Quinn * Luis Rosario * Jessica H. Stark * Volkan Granit * Hannah Briemberg * Sneha Chenji * Annie Dionne * Angela Genge * Wendy Johnston * Lawrence Korngut * Christen Shoesmith * Lorne Zinman * the Canadian ALS Neuroimaging Consortium (CALSNIC) * Sanjay Kalra * Michael Benatar ## Abstract **Importance** Amyotrophic lateral sclerosis (ALS) is a multi-system disorder characterized primarily by motor neuron degeneration, but may be accompanied by cognitive dysfunction. Statistically appropriate criteria for establishing cognitive impairment (CI) in ALS are lacking. **Objective** Define thresholds for CI in ALS using quantile regression (QR) that accounts for age and education in a North American (NAmer) cohort. **Design** QR of cross-sectional data from a multi-center NAmer cohort of healthy adults was used to model the 5th percentile of cognitive scores on the Edinburgh Cognitive and Behavioral ALS Screen (ECAS). The QR approach was compared to a traditional 2 standard deviation (SD) cut-off approach using the same NAmer cohort (2SD-NAmer) and to existing UK-based normative data derived using the 2SD approach (2SD-UK) to assess the impact of cohort selection and statistical model in identifying CI ALS patients. **Participants** 269 healthy adults from NAmer, recruited by the University of Pennsylvania (PENN; N=82), the University of Miami through the CRiALS study (CRiALS; N=40), and the Canadian ALS Neuroimaging Consortium (CALSNIC; N=147) were included to establish ECAS thresholds for defining CI. We then evaluated the frequency of CI in 182 ALS patients from PENN. **Main Outcomes** We defined two new sets of normative thresholds, based on NAmer heathy adult performance, for each ECAS domain score and the composite scores using QR and 2SD statistical approaches. We then applied the 2SD-NAmer and QR-NAmer, as well as the previously established and widely-used 2SD-UK, thresholds to evaluate the frequency of CI in ALS patients. **Results** QR-NAmer models revealed that increased age and reduced educational attainment negatively impact cognitive performance on the ECAS. Based on the QR-NAmer normative cutoffs, the prevalence of CI in the 182 PENN ALS patients was 15.9% for ECAS ALS-Specific and 15.4% for ECAS Total. These estimates are more conservative than estimates ranging from 15.4%-34.6% impaired based on 2SD approaches. **Conclusions and Relevance** This report establishes normative thresholds for using ECAS to identify whether ALS patients in the NAmer population have CI. The choice of statistical method and normative cohort has a substantial impact on defining CI in ALS. **Question** How to define cognitive impairment (CI) in amyotrophic lateral sclerosis (ALS) using the Edinburgh Cognitive and Behavioral ALS Screen (ECAS)? **Findings** Age- and education-adjusted quantile regression (QR) yields thresholds for defining CI that differ meaningfully from those derived from parametric methods without age- and education-adjustment. Thresholds also differ between UK and North American cohorts. Applying our North American-based QR norms to an American ALS cohort at a single center identified CI based on ECAS performance in ∼16% patients, compared to 15.4%-34.6% patients using other approaches. **Meaning** The choice of statistical method and normative cohort has a substantial impact on defining CI in ALS. Amyotrophic lateral sclerosis (ALS) is a multi-system disorder primarily characterized by progressive lower motor neuron (LMN) and upper motor neuron (UMN) degeneration1, but there is also increasing evidence for cognitive and behavioral impairment2. Indeed, current criteria for ALS frontotemporal spectrum disorder (ALS-FTSD) recommend two axes for the diagnosis of ALS including (I) defining the motor neuron syndrome and (II) defining neuropsychological deficits**3**. Cognitive impairment in ALS is an important clinical consideration given its association with poor prognosis including reduced survival and functional decline**4**. The Edinburgh Cognitive and Behavioral Assessment Screen (ECAS) is an established brief neuropsychological assessment designed specifically for ALS that accommodates physical challenges (e.g., limited ability to speak or write) that potentially confound traditional neuropsychological testing**5**. Administration of the ECAS takes approximately 25 minutes to complete and uses a 136-point scale to evaluate several domains of cognition including language, executive functioning, and verbal fluency that are commonly believed to be most often affected in ALS, collectively contributing to an “ALS-Specific” composite score; as well as visuospatial and memory domains that are thought to be less frequently affected in ALS and contribute to an “ALS-Nonspecific” score. The ECAS has been validated against more extensive neuropsychological tests**6**,**7** and multimodal studies, suggesting that ECAS performance relates to biologically plausible regional neuroimaging8 and neuropathological burden9 in ALS patients. While the ECAS has been translated into several languages10-13, normative data to empirically define cognitive impairments in North American ALS patients are lacking, and only rare prior studies have considered age and education when defining thresholds of cognitive impairment14,15. Moreover, prior approaches to defining impairment cutoffs on the ECAS have typically been based on relatively small series of healthy control data and have been based on statistical models that may violate important assumptions about the psychometric properties of ECAS scores. In this report we present and compare 3 sets of normative thresholds for defining cognitive impairment, calculated based on different reference populations (N=40 single-center United Kingdom controls vs. N=269 multi-center North American controls) and statistical methods: the most common approach of “2 standard deviations below the mean” (2SD) vs. quantile regression with covariate adjustment (QR), as summarized in Table 1. Moreover, we illustrate how the choice of normative cutoffs impacts the classification of impairment (or not) in a cohort of 182 ALS patients. View this table: [Table 1.](http://medrxiv.org/content/early/2020/12/26/2020.12.23.20248410/T1) Table 1. Study populations from which normative data were collected, statistical methods, and threshold strategies used to define cognitive impairment on the ECAS ## Methods ### Control Participants and Sources of Normative Data Data to establish normative ECAS performance were pooled together from three North American sources: the research cohort at the Penn Frontotemporal Degeneration Center and Comprehensive ALS Center at the University of Pennsylvania (PENN); the multi-center study conducted by the Clinical Research in ALS Biomarker study at the University of Miami (CRiALS); and the multi-center study conducted by the Canadian ALS Neuroimaging Consortium (CALSNIC) 16. This resulted in a total of 269 healthy adults; each source of data is described in detail below and demographic characteristics are summarized in Table 1. #### PENN 82 community-dwelling healthy adults who self-report no history of neurological or significant psychiatric condition and were screened to have a Mini Mental Status Examination score of 27-30. The PENN research protocol was approved by an Institutional Review Board convened at the University of Pennsylvania and all participants provided written consent following an approved informed consent procedure. #### CRiALS 40 healthy adults recruited from across the U.S. with no history of neurological disease, no significant psychiatric history, and either have no family history of ALS or FTD or do not carry the genetic mutation(s) known to cause ALS/FTD in their family. The CRiALS research protocol was approved by an Institutional Review Board convened at the University of Miami and all participants provided written consent following an approved informed consent procedure. #### CALSNIC 147 healthy adults recruited across Canada and the U.S. and with no history of neurological disease, head trauma, or significant psychiatric history. The CALSNIC research protocol was approved by a Health Research Ethics Board convened at each of the participating universities including University of Alberta, University of British Columbia, Université Laval, University of Calgary, the Montreal Neurological Institute and Hospital at McGill University, Western University, and University of Toronto. All participants at each of these institutions provided written consent following an approved informed consent procedure. Education was recorded differently across studies, and for simplicity in interpretation, we categorically defined education for statistical models as individuals who completed a four-year college degree or higher level of education (≥ College) or did not (< College). ### ALS Participants To illustrate the impact of different ECAS normative cutoffs on identifying cognitive dysfunction in ALS, we evaluated 182 ALS patients at PENN who were diagnosed by a board-certified neurologist according to published consensus criteria1. Demographic and clinical characteristics are described in Online Supplementary Table 1. In addition to a clinical diagnosis of ALS, further inclusion criteria included completion of the ECAS along with three clinical assessments comprised of (1) a seated forced vital capacity (FVC) assessment of respiratory function; (2) the Revised ALS Functional Rating Scale (ALSFRS-R); and (3) Penn Upper Motor Neuron Score (PUMNS). All patients participated in an informed consent procedure approved by an Institutional Review Board convened at the University of Pennsylvania. View this table: [Online Supplementary Table 1.](http://medrxiv.org/content/early/2020/12/26/2020.12.23.20248410/T4) Online Supplementary Table 1. Summary demographics and ECAS performance for PENN ALS Patients (N=182) ### ECAS All healthy adult control participants and ALS patients completed the North American ECAS (Version 1) using published guidelines and instructions ([https://ecas.psy.ed.ac.uk/ecas-international/#American](https://ecas.psy.ed.ac.uk/ecas-international/#American)) as described in detail elsewhere5. All evaluators were trained in the administration and scoring of the ECAS, which assesses the participants’ oral response performance in each of the 5 domains (language, fluency, executive, memory, and visuospatial function). ### Statistical Approaches All statistical analyses were performed using R software (Version 3.5.1). Visual assessment of histograms and Q-Q plots for each domain as well as the composite scores on the ECAS suggested that most scores are not normally distributed (see Online Supplementary Figure 1) and Kolmogorov-Smirnov tests relative to a normal distribution centered on the mean of each score confirmed lack of normality for each domain (all p<0.1e-8). Therefore, we compare the most common approaches for defining cutoffs based on 2 standard deviations (2SD) below the mean across both sources of normative data relative to a Quantile Regression (QR) approach that does not make Gaussian assumptions and further accommodates adjustments for age and education. ![Online Supplementary Figure 1.](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2020/12/26/2020.12.23.20248410/F3.medium.gif) [Online Supplementary Figure 1.](http://medrxiv.org/content/early/2020/12/26/2020.12.23.20248410/F3) Online Supplementary Figure 1. Histograms (top) and Quantile-Quantile (bottom) plots of the distribution of healthy adult performance on each ECAS domain and composite subscore. Note. In each histogram the solid red line represents mean performance and the dotted red line represents 2 standard deviations below the mean, selected to represent the threshold for impairment employed using the 2SD-NAmer approach. We additionally summarize descriptive results using both parametric (e.g., mean) and non-parametric (e.g., median) statistics to facilitate comparison to previous reports. ### Methods for Defining Thresholds of Cognitive Impairment #### Parametric UK Norms (2SD-UK) The most widely applied approach for defining cognitive impairment on the ECAS is based on a previously established < -2 standard deviation cutoff relative to the mean in each domain and composite subscore from a sample of 40 Edinburgh-based healthy adults5. #### Parametric North American (2SD-NAmer) For comparison purposes we also implemented a < -2 standard deviation cutoff relative to the mean in each domain and the composite subscores, but using our sample of 269 North American healthy adults. #### Quantile Regression North American (QR-NAmer) Quantile regression has one key benefit over traditional parametric approaches for defining normative cutoffs14,15. It makes no assumption about the data distribution and is, therefore, more appropriate for skewed data distributions that may have floor or ceiling effects. In addition, it can account for heteroscedasticity in which the variance of an outcome (e.g., ECAS score) is not uniform across a factor (e.g., age or education), whereas the 2SD method does not allow for covariate adjustments except through stratification17. QR-NAmer cutoffs were calculated using the R quantregGrowth package18 for the 5th percentile of performance in each ECAS domain and composite subscores for the 269 healthy adults. ## Results ### Normative ECAS Performance in Healthy Adults The study population of 269 healthy adults (60% with at least a college education) included 150 females (56%) and ranged in age from 24 to 83, with a mean ± SD of 55.5 ± 11.5 years (Table 2). Median [interquartile range] of ECAS Total, ALS Specific and ALS Non-Specific scores were 115 [110-120], 87 [81-90] and 29 [27-31], with maximum possible scores of 136, 100 and 36 respectively (Table 2). Age- and education-adjusted estimates of 5th percentile thresholds for cognitive impairment are summarized for all ECAS scores, including each cognitive domain, in Table 3 (see also Figure 1 and Online Supplementary Figure 2). Importantly, the QR-NAmer approach demonstrates a significant decline in healthy adult performance associated with age on each composite score (Figure 1, Table 3), with higher educational attainment (completing college) increasing the threshold for cognitive impairment by 1-4 points. Moreover, for individual cognitive domains, the 5th percentile of performance also showed a slope associated with age for memory, fluency, executive and language performance, but not the visuospatial domain in which performance approached ceiling across all ages (Online Supplementary Figure 2). ![Online Supplementary Figure 2.](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2020/12/26/2020.12.23.20248410/F4.medium.gif) [Online Supplementary Figure 2.](http://medrxiv.org/content/early/2020/12/26/2020.12.23.20248410/F4) Online Supplementary Figure 2. Quantile regression model for Language, Fluency, Executive, Memory, and Visuospatial domains, including 5th Percentile cutoff (red) as well as for comparison the two standard deviation approach for North America (2SD-NAmer; orange) and United Kingdom (2SD-NAmer; blue). View this table: [Table 2.](http://medrxiv.org/content/early/2020/12/26/2020.12.23.20248410/T2) Table 2. Summary demographics and performance on the Edinburgh Cognitive & Behavioral Screen (ECAS) from three cohort studies of healthy adults (N=269) from North America. View this table: [Table 3.](http://medrxiv.org/content/early/2020/12/26/2020.12.23.20248410/T3) Table 3. Summary of ECAS cutoffs including two standard deviation approach with published United Kingdom values (2SD-UK) 5 and in the North American cohort using a two standard deviation (2SD-NAmer) and Quantile Regression (QR-NAmer) statistical approaches. ![Figure 1.](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2020/12/26/2020.12.23.20248410/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2020/12/26/2020.12.23.20248410/F1) Figure 1. Quantile regression model for ECAS Total, ALS Specific, and ALS Non-Specific Scores across age, including 5th Percentile cutoff (red) as well as for comparison the two standard deviation approach for North America (2SD-NAmer; orange) and United Kingdom (2SD-NAmer; blue). ### Prevalence of Cognitive Impairments in PENN ALS Cohort The prevalence of cognitive impairment in ALS varies depending on the population used to generate normative data as well as the approach used to determine thresholds for defining cognitive impairment (Figure 2 and Online Supplementary Table 2). When comparing the statistical approaches (2SD and QR) developed across NAmer and UK cohorts, the 2SD-NAmer and QR-NAmer cutoffs suggest that ∼16% of PENN ALS patients exhibit cognitive impairment based on the ALS-Specific and ECAS Total composite scores, substantially fewer than that estimated using the 2SD-UK approach for ALS-Specific (32.4% impaired) or ECAS Total (34.6% impaired). The discordance between cutoff approaches is even more apparent in the Language domain with an estimated ∼63% impairment relative to ∼19% and ∼20% impaired in the PENN ALS cohort based on the 2SD-NAmer and QR-NAmer approaches, respectively. Fluency and Executive domains also suggest a lower frequency of cognitive impairment in the PENN ALS cohort using the 2SD-NAmer and QR-NAmer approach relative to the 2SD-UK approach. View this table: [Online Supplementary Table 2.](http://medrxiv.org/content/early/2020/12/26/2020.12.23.20248410/T5) Online Supplementary Table 2. Frequency (%) of impairment in 182 ALS based on cutoffs from two standard deviation cutoff approach from Abrahams et al. (2014, 2SD-UK), North American cohort (2SD-NAmer) cutoffs, and Quantile Regression North American (QR-NAmer) cutoffs. ![Figure 2.](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2020/12/26/2020.12.23.20248410/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2020/12/26/2020.12.23.20248410/F2) Figure 2. Prevalence of impairment on the ECAS in 182 ALS patients using three different sets of normative cutoffs. Note. QR-NAmer=Quantile regression North American; 2SD-NAmer=2 standard deviation cutoff North American; 2SD-UK= 2 standard deviation cutoff United Kingdom In contrast to the NAmer vs. UK comparisons above, the differences between the QR and parametric statistical approaches yield relatively similar estimates when evaluating of the prevalence of cognitive impairment in the PENN ALS cohort using NAmer reference populations. These include the ALS Specific score - 15.9% (QR-NAmer) vs. 16.5% (2SD-NAmer) and the individual cognitive domains that contribute to this score: Language - 19.8% (QR-NAmer) vs. 19.2% (2SD-NAmer); Executive - 11.0% (QR-NAmer) vs. 11.5% (2SD-NAmer); and Fluency - 9.3% (QR-NAmer) vs. 14.8% (2SD-NAmer). The greatest divergence between QR-NAmer and 2SD-NAmer approaches was for Visuospatial impairment, 22.5% (QR-NAmer) vs. 32.4% (2SD-NAmer), that together with Memory (QR-NAmer=8.8% vs. 2SD-NAmer=9.9%) contributes to a discrepancy in rates of impairment on the ALS Non-Specific score - 12.6% (QR-NAmer) vs. 14.3% (2SD-NAmer). ## Discussion Increasing interest in frontotemporal spectrum impairment in ALS and the overlap between ALS and FTD, have necessitated development of tools to quantify the nature and frequency of cognitive dysfunction in ALS. While the ECAS, a brief assessment tool, has emerged as the dominant instrument for evaluating cognition and behavior in ALS19, limited reference data (at least within North America) has hampered progress. In particular, efforts are underway to better understand genotype-phenotype relationships, including predictors of cognitive impairment20. Without appropriate normative data these efforts may be less successful due to potentially significant misclassification of cognitive impairment. Moreover, the recently revised Airlie House guidelines for the design and implementation of ALS clinical trials recommend that investigators stratify for cognitive impairments21; it is therefore essential to appropriately and accurately define whether individual patients are impaired or not. Here we present normative data for the North American ECAS based on a moderately-sized, multi-center dataset and recommend (a) the use of quantile regression as appropriate for estimating normative thresholds (e.g. 5th percentile) of data that are not normally distributed, and (b) adjusting for age and education, factors known to associate with cognitive function. Applying these findings to the ALS patient cohort at PENN, we have estimated the prevalence of cognitive impairment in ALS to be ∼16%. This is substantially less frequent than what we and others have previously estimated based on previously published norms, which suggested typically up to ∼50%3,22, and one estimate as high as 70%23, of ALS patients have cognitive impairment, though these prior estimates that are inflated in comparison to our observations were derived from neuropsychological tests that, unlike ECAS, do not control for motor confounds. Future work will be necessary to validate the QR-NAmer approach by comparing these suggested cutoffs relative to other more extensive neuropsychological assessments, but at present this is challenging because many other validated neuropsychological assessments are typically confounded by motor features. While there is increasing evidence for language impairment in ALS, the current findings suggest that estimates of the rate of language impairments vary widely depending on the statistical approach and control population employed to generate normative data, ranging from ∼19% for the 2SD-NAmer and QR-NAmer approaches to as high as ∼63% of PENN ALS patients using the 2SD-UK approach. These findings therefore emphasize that sociodemographic differences across cultures are a critical consideration when defining normal language performance. As the ECAS becomes increasingly used internationally, it may be valuable to account for potential cultural and regional confounds in subsequent versions of the test, such as including cross-linguistic naming of objects 24, differences in orthographic-phonemic mapping that may influence spelling across languages, and implementing norms across several languages25. While letter-guided verbal fluency constitutes a “domain” distinct from language in the ECAS, it is linguistically-mediated and thus susceptible to cross-linguistic differences, such as variable letter frequency26. It will, therefore, likely be important to re-calibrate the “verbal fluency indices” across ECAS versions. Nevertheless, it is valuable to note the overall relatively high rate of impairment on language skills probed by ECAS. This has been noted by other investigators27, but may be underappreciated. Lastly, while semantic deficits have been reported in ALS28, there is increasing evidence for grammatical and discourse deficits in ALS 29-31. Supplementing the current language domain with tests of grammar and discourse in subsequent ECAS versions may provide a more accurate picture of language impairment in ALS. Our use of the QR-NAmer approach, in addition to accounting for non-normality, benefits by also accounting for cognitive performance across age. Importantly, ECAS Total score cutoffs range from 106 to 87 points depending on the age of the individual, highlighting the importance of considering age in the determination of impairment by using a dynamic rather than static cutoff strategy. Memory appears to be the domain most affected by age, as well as the ALS Non-Specific score, which suggests that these components are more sensitive to age-related or non-FTD contributors to cognitive decline. Indeed, memory decline in aging may reflect mild cognitive impairment or prodromal Alzheimer disease (AD)32, and AD co-pathology has been noted in autopsy-confirmed series of ALS patients sensitive to cognitive impairment33. In this context, it is important to consider that ALS cognitive impairment (ALSci), as defined using current clinical criteria3, may be associated with the ALS-FTSD spectrum as a prodromal phase of ALS-FTD or ALS-Dementia. Importantly, the QR-NAmer approach suggests that the highest frequency of impairment based on composite scores (e.g., ALS Specific, ALS Non-Specific, Total) was the ALS Specific composite, which comprises the more predominantly affected executive, fluency, and language domains. Beyond adjusting for age, the level of educational attainment has a modest impact on normative cutoff values. In particular, attainment of a college degree appears to have the greatest impact on verbal fluency and executive domains that contribute to the ALS Specific score, and only a 1-point impact on memory domains, while language and visuospatial performance remain constant across education groups. There are at least two possible reasons for this observation. One possibility for a lack of contribution of education to language and visuospatial domains may simply be a function of these domains having the most limited statistical variance. Another possibility, mentioned above, is that the limited range of probed language areas may be relatively insensitive to education. Yet an alternative possibility is that verbal fluency, executive, and memory domains are indeed preferentially impacted by educational status. For example, models of cognitive reserve suggest that factors like education may help stave off the clinical manifestations of underlying pathologies in aging individuals34 and evidence suggests that reserve may selectively impact executive functions but not visuospatial functions in healthy aging35 and frontotemporal degeneration36. There are several caveats to consider. First, behavior is an important component of ALS-FTSD and, while this report establishes the frequency of cognitive impairment in ALS, it does not establish normative data for defining behavioral impairment in healthy adults or ALS patients. Second, our estimates of normative cognitive performance are based on a cross-sectional evaluation, which represents a snapshot of a progressive disease in which cognitive impairment may only emerge as disease unfolds37. While initial efforts have parametrically defined longitudinal change on the ECAS for healthy controls38, it is critical for future longitudinal investigations to define age-related rate of cognitive decline in healthy controls and in a manner that accounts for sociodemographic and cultural differences, in order to determine how the tempo of disease progression in ALS differs from normal age-related cognitive decline. Third, a limitation of pooling normative healthy adult data across cohorts is that educational attainment was not captured in a harmonized manner and therefore we were limited to dichotomizing education as college completion vs. less than a college education. However, cognitive performance may be more sensitive to finer-grained educational differences (e.g., high school, graduate degree) and this is an important consideration for future investigations. Likewise, our control cohort had a greater frequency of college completion (∼65%) than the general population; although the current study is the largest ECAS normative study to date, future studies with increased sampling of the general population may better capture the influence of education on ECAS performance. Lastly, we defined thresholds of cognitive impairment using a commonly implemented 5th percentile cutoff for the QR approach and 2SD cutoff for the other approaches, but it is possible that alternative less conservative thresholds may be more appropriate. In summary, estimates of the frequency of cognitive impairment in ALS vary widely depending on the cohort and statistical approach used to define thresholds for defining cognitive impairment. While our estimates are based on application of norms to ALS patients in the PENN cohort, we suggest that the use of quantile regression models, as employed in our QR-Namer approach, provides a robust strategy to define cognitive impairments that is statistically appropriate and accounts for potential confounding factors such as age and education. ## Data Availability Data may be provided to outside investigators pending review of reasonable requests. ## Declaration of Conflicts Dr. McMillan receives research funding outside the submitted work from Biogen, Inc and provides consulting services for personal fees from Invicro and Axon Advisors on behalf of Translational Bioinformatics, LLC. He also receives an honorarium as Associate Editor of NeuroImage: Clinical. Dr. Johnston receives research funding from Annexon, Alexion, ALS-Pharma, Biogen, Cytokinetics, Mallinckrodt, Mitsubishi-Tanabe Canada, Orion, and serves as a paid consultant to Biogen and Mitsubishi-Tanabe Canada. Dr. Genge provides consulting services for the following companies: AB Sciences, Akcea, Alexion, AL-S Pharma, Anavex, Avexis, Bayshore, Biogen, Clene, CSL Behring, Cytokinetics, QurAlis, Mitsubishi Tenabe Pharma America, Novartis, Orion, Revalesio, Roche, Sanofi Genzyme, and Wave life sciences. She is also CRU Medical Director, PI or sub-PI on trials sponsored by the following companies: AB Sciences, AL-S Pharma, Acceleron, Amicus, Alnylam, Bioblast, Biogen, BMS, Boston Biomedical, Cytokinetics, Sanofi Genzyme, Grifols, Ionis, Lily, Mallinckrodt, Medimmune, Novartis, Orion, Orphazyme, Pfizer, Ra Pharmaceuticals, Roche, Teva, and UCB. Dr. Benatar reports personal fees from Roche and Biogen. Drs. Wuu, Rascovsky, Cosentino, Grossman, Elman, Quinn, Rosario, Stark, Granit, Briemberg, Chenji, Dionne, Korngut, Shoesmith, Zinman report no conflicts of interest. ## Acknowledgments We extend thanks to research coordinators at the University of Pennsylvania including Laura Baehr, Laura Hennessy, and Hayley Donaldson for their efforts with participant recruitment and evaluation. CALSNIC additionally thanks the research team for participant recruitment and evaluations. Additionally, CRiALS extends thanks to the research team at the University of Miami for participant recruitment and evaluation (Anne-Laure Grignon, Danielle Dauphin, Danielle Sheldon, Sumaira Hussain, Anne Cooley, Jessica Stark, Jessica Medina, Ashley Manso). Dr. McMillan reports grants from the National Institutes of Health (NIH; AG017586; AG066597). Dr. Kalra reports grants that support CALSNIC including the Canadian Institutes of Health Research, ALS Society of Canada, Brain Canada Foundation, and Shelly Mrkonjic ALS Research Fund. Dr. Benatar reports grants from Muscular Dystrophy Association (Grants #4365 and #172123), the ALS Association (Grant #2015), and the National Institutes of Health (NIH; NS105479); philanthropic support from the ALS Recovery Fund and the Kimmelman Estate during the conduct of the study; * Received December 23, 2020. * Revision received December 23, 2020. * Accepted December 25, 2020. * © 2020, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. 1.Brooks BR, Miller RG, Swash M, Munsat TL, World Federation of Neurology Research Group on Motor Neuron Diseases. El Escorial revisited: revised criteria for the diagnosis of amyotrophic lateral sclerosis. In: Vol 1. 2000:293–299. 2. 2.Goldstein LH, Abrahams S. Changes in cognition and behaviour in amyotrophic lateral sclerosis: nature of impairment and implications for assessment. Lancet Neurol. 2013;12(4):368–380. doi:10.1016/S1474-4422(13)70026-7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1474-4422(13)70026-7&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23518330&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000318531600013&link_type=ISI) 3. 3.Strong MJ, Abrahams S, Goldstein LH, et al. Amyotrophic lateral sclerosis - frontotemporal spectrum disorder (ALS-FTSD): Revised diagnostic criteria. Amyotroph Lateral Scler Frontotemporal Degener. January 2017:1–22. doi:10.1080/21678421.2016.1267768. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/21678421.2016.1267768&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) 4. 4.Elamin M, Bede P, Byrne S, et al. Cognitive changes predict functional decline in ALS: a population-based longitudinal study. Neurology. 2013;80(17):1590–1597. doi:10.1212/WNL.0b013e31828f18ac. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1212/WNL.0b013e31828f18ac&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23553481&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) 5. 5.Abrahams S, Newton J, Niven E, Foley J, Bak TH. Screening for cognition and behaviour changes in ALS. Amyotroph Lateral Scler Frontotemporal Degener. 2014;15(1-2):9–14. doi:10.3109/21678421.2013.805784. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3109/21678421.2013.805784&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23781974&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) 6. 6.Niven E, Newton J, Foley J, et al. Validation of the Edinburgh Cognitive and Behavioural Amyotrophic Lateral Sclerosis Screen (ECAS): A cognitive tool for motor disorders. Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(3-4):172–179. doi:10.3109/21678421.2015.1030430. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3109/21678421.2015.1030430&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25967542&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) 7. 7. De Icaza Valenzuela Mónica M, H Bt, Suvankar P, Sharon A. The Edinburgh Cognitive and Behavioral ALS screen: relationship to age, education, IQ and the Addenbrooke’s Cognitive Examination-III. Amyotroph Lateral Scler Frontotemporal Degener. 2018;00(00):1–7. doi:10.1080/21678421.2018.1491601. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/21678421.2018.1491601&link_type=DOI) 8. 8.Trojsi F, Caiazzo G, Siciliano M, et al. Microstructural correlates of Edinburgh Cognitive and Behavioural ALS Screen (ECAS) changes in amyotrophic lateral sclerosis. Psychiatry Res Neuroimaging. 2019;288:67–75. doi:10.1016/j.pscychresns.2019.04.001. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.pscychresns.2019.04.001&link_type=DOI) 9. 9.Gregory JM, McDade K, Bak TH, et al. Executive, language and fluency dysfunction are markers of localised TDP-43 cerebral pathology in non-demented ALS. Journal of Neurology, Neurosurgery, and Psychiatry. September 2019:jnnp–2019–320807. doi:10.1136/jnnp-2019-320807. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiam5ucCI7czo1OiJyZXNpZCI7czo4OiI5MS8yLzE0OSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzEyLzI2LzIwMjAuMTIuMjMuMjAyNDg0MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 10. 10.Bakker LA, Schröder CD, Spreij LA, et al. Derivation of norms for the Dutch version of the Edinburgh cognitive and behavioral ALS screen. Amyotroph Lateral Scler Frontotemporal Degener. 2019;20(1-2):19–27. doi:10.1080/21678421.2018.1522352. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/21678421.2018.1522352&link_type=DOI) 11. 11.Lulé D, Burkhardt C, Abdulla S, et al. The Edinburgh Cognitive and Behavioural Amyotrophic Lateral Sclerosis Screen: a cross-sectional comparison of established screening tools in a German-Swiss population. Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(1-2):16–23. doi:10.3109/21678421.2014.959451. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3109/21678421.2014.959451&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25292386&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) 12. 12.Mora JS, Salas T, Fernández MC, et al. Spanish adaptation of the edinburgh cognitive and behavioral amyotrophic lateral sclerosis screen (ECAS). Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(1-2):74–79. doi:10.1080/21678421.2017.1406952. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/21678421.2017.1406952&link_type=DOI) 13. 13.Poletti B, Solca F, Carelli L, et al. Cognitive-behavioral longitudinal assessment in ALS: the Italian Edinburgh Cognitive and Behavioral ALS screen (ECAS). Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(5-6):387–395. doi:10.1080/21678421.2018.1473443. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/21678421.2018.1473443&link_type=DOI) 14. 14.Loose M, Burkhardt C, Aho-Özhan H, et al. Age and education-matched cut-off scores for the revised German/Swiss-German version of ECAS. Amyotroph Lateral Scler Frontotemporal Degener. 2016;17(5-6):374–376. doi:10.3109/21678421.2016.1162814. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3109/21678421.2016.1162814&link_type=DOI) 15. 15.Pinto-Grau M, Burke T, Lonergan K, et al. Screening for cognitive dysfunction in ALS: validation of the Edinburgh Cognitive and Behavioural ALS Screen (ECAS) using age and education adjusted normative data. Amyotroph Lateral Scler Frontotemporal Degener. 2017;18(1-2):99–106. doi:10.1080/21678421.2016.1249887. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/21678421.2016.1249887&link_type=DOI) 16. 16.Kalra S, Khan MU, Barlow L, et al. The Canadian ALS Neuroimaging Consortium (CALSNIC) - a multicentre platform for standardized imaging and clinical studies in ALS. medRxiv. August 2020:1–19. doi:10.1101/2020.07.10.20142679. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wNy4xMC4yMDE0MjY3OXYyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMTIvMjYvMjAyMC4xMi4yMy4yMDI0ODQxMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 17. 17.Benatar M, Wuu J, Peng L. Reference data for commonly used sensory and motor nerve conduction studies. Muscle Nerve. 2009;40(5):772–794. doi:10.1002/mus.21490. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/mus.21490&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19827092&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000271174400004&link_type=ISI) 18. 18.Muggeo VMR, Sciandra M, Tomasello A, Calvo S. Estimating growth charts via nonparametric quantile regression: a practical framework with application in ecology. Environ Ecol Stat. 2013;20(4):519–531. doi:10.1007/s10651-012-0232-1. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s10651-012-0232-1&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000327086000001&link_type=ISI) 19. 19.Hodgins F, Mulhern S, Abrahams S. The clinical impact of the Edinburgh Cognitive and Behavioural ALS Screen (ECAS) and neuropsychological intervention in routine ALS care. Amyotroph Lateral Scler Frontotemporal Degener. 2019;12:1–8. doi:10.1080/21678421.2019.1674874. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/21678421.2019.1674874&link_type=DOI) 20. 20.Placek K, Benatar M, Wuu J, et al. Machine learning suggests polygenic risk for cognitive dysfunction in amyotrophic lateral sclerosis. EMBO Mol Med. 2020;15(1):e12595. doi:10.15252/emmm.202012595. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15252/emmm.202012595&link_type=DOI) 21. 21.van den Berg LH, Sorenson E, Gronseth G, et al. Revised Airlie House consensus guidelines for design and implementation of ALS clinical trials. Neurology. 2019;92(14):e1610–e1623. doi:10.1212/WNL.0000000000007242. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibmV1cm9sb2d5IjtzOjU6InJlc2lkIjtzOjExOiI5Mi8xNC9lMTYxMCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzEyLzI2LzIwMjAuMTIuMjMuMjAyNDg0MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 22. 22.Ringholz GM, Appel SH, Bradshaw M, Cooke NA, Mosnik DM, Schulz PE. Prevalence and patterns of cognitive impairment in sporadic ALS. Neurology. 2005;65(4):586–590. doi:10.1212/01.wnl.0000172911.39167.b6. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibmV1cm9sb2d5IjtzOjU6InJlc2lkIjtzOjg6IjY1LzQvNTg2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMTIvMjYvMjAyMC4xMi4yMy4yMDI0ODQxMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 23. 23.Frank B, Haas J, Heinze HJ, Stark E, Münte TF. Relation of neuropsychological and magnetic resonance findings in amyotrophic lateral sclerosis: evidence for subgroups. Clin Neurol Neurosurg. 1997;99(2):79–86. doi:10.1016/s0303-8467(96)00598-7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0303-8467(96)00598-7&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9213049&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1997XG39100001&link_type=ISI) 24. 24.Ardila A. Toward the development of a cross-linguistic naming test. Arch Clin Neuropsychol. 2007;22(3):297–307. doi:10.1016/j.acn.2007.01.016. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.acn.2007.01.016&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17303376&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) 25. 25.Bates E, D’Amico S, Jacobsen T, et al. Timed picture naming in seven languages. Psychon Bull Rev. 2003;10(2):344–380. doi:10.3758/bf03196494. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3758/BF03196494&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12921412&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) 26. 26.Rosselli M, Ardila A, Salvatierra J, Marquez M, Matos L, Weekes VA. A crosslinguistic comparison of verbal fluency tests. Int J Neurosci. 2002;112(6):759–776. doi:10.1080/00207450290025752. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/00207450290025752&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12325314&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000175943900008&link_type=ISI) 27. 27.Taylor LJ, Brown RG, Tsermentseli S, et al. Is language impairment more common than executive dysfunction in amyotrophic lateral sclerosis? Journal of Neurology, Neurosurgery, and Psychiatry. 2013;84(5):494–498. doi:10.1136/jnnp-2012-303526. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiam5ucCI7czo1OiJyZXNpZCI7czo4OiI4NC81LzQ5NCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzEyLzI2LzIwMjAuMTIuMjMuMjAyNDg0MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 28. 28.Libon DJ, McMillan C, Avants B, et al. Deficits in concept formation in amyotrophic lateral sclerosis. 2012;26(4):422–429. doi:10.1037/a0028668. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1037/a0028668&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22612577&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) 29. 29.Ash S, Olm C, McMillan CT, et al. Deficits in sentence expression in amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(1-2):31–39. doi:10.3109/21678421.2014.974617. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3109/21678421.2014.974617&link_type=DOI) 30. 30.Ash S, Menaged A, Olm C, et al. Narrative discourse deficits in amyotrophic lateral sclerosis. Neurology. 2014;83(6):520–528. doi:10.1212/WNL.0000000000000670. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibmV1cm9sb2d5IjtzOjU6InJlc2lkIjtzOjg6IjgzLzYvNTIwIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMTIvMjYvMjAyMC4xMi4yMy4yMDI0ODQxMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 31. 31.Nevler N, Ash S, McMillan C, et al. Automated analysis of natural speech in amyotrophic lateral sclerosis spectrum disorders. Neurology. 2020;95(12):e1629–e1639. doi:10.1212/WNL.0000000000010366. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibmV1cm9sb2d5IjtzOjU6InJlc2lkIjtzOjExOiI5NS8xMi9lMTYyOSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzEyLzI2LzIwMjAuMTIuMjMuMjAyNDg0MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 32. 32.Knopman DS, Caselli RJ. Appraisal of cognition in preclinical Alzheimer’s disease: a conceptual review. Neurodegener Dis Manag. 2012;2(2):183–195. doi:10.2217/nmt.12.5. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2217/nmt.12.5&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22798965&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) 33. 33.Brettschneider J, Libon DJ, Toledo JB, et al. Microglial activation and TDP-43 pathology correlate with executive dysfunction in amyotrophic lateral sclerosis. Acta Neuropathol. 2012;123(3):395–407. doi:10.1007/s00401-011-0932-x. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00401-011-0932-x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22210083&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000301856300006&link_type=ISI) 34. 34.Stern Y. Cognitive reserve. Neuropsychologia. 2009;47(10):2015–2028. doi:10.1016/j.neuropsychologia.2009.03.004. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.neuropsychologia.2009.03.004&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19467352&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F12%2F26%2F2020.12.23.20248410.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000267189200001&link_type=ISI) 35. 35.Lavrencic LM, Churches OF, Keage HAD. Cognitive reserve is not associated with improved performance in all cognitive domains. Appl Neuropsychol Adult. 2017;25(5):473–485. doi:10.1080/23279095.2017.1329146. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/23279095.2017.1329146&link_type=DOI) 36. 36.Placek K, Massimo L, Olm C, et al. Cognitive reserve in frontotemporal degeneration: Neuroanatomic and neuropsychological evidence. Neurology. 2016;87(17):1813–1819. doi:10.1212/WNL.0000000000003250. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibmV1cm9sb2d5IjtzOjU6InJlc2lkIjtzOjEwOiI4Ny8xNy8xODEzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMTIvMjYvMjAyMC4xMi4yMy4yMDI0ODQxMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 37. 37.Crockford C, Newton J, Lonergan K, et al. ALS-specific cognitive and behavior changes associated with advancing disease stage in ALS. Neurology. 2018;91(15):e1370–e1380. doi:10.1212/WNL.0000000000006317. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibmV1cm9sb2d5IjtzOjU6InJlc2lkIjtzOjExOiI5MS8xNS9lMTM3MCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzEyLzI2LzIwMjAuMTIuMjMuMjAyNDg0MTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 38. 38.Crockford C, Newton J, Lonergan K, et al. Measuring reliable change in cognition using the Edinburgh Cognitive and Behavioural ALS Screen (ECAS). Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(1-2):65–73. doi:10.1080/21678421.2017.1407794. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/21678421.2017.1407794&link_type=DOI)