Abstract
Eye tracking is one of the emerging techniques clinicians use to establish the presence and monitor the progression of neurodegenerative disorders (NDs). The clinical observation and assessment of extra-ocular movements is common practice to detect motoric and cognitive impairments but remains observer-dependent and subjective. In the present study, we propose an algorithm that can automatically identify saccades, fixation, smooth pursuit, and blinks using a non-invasive eye-tracker and, subsequently, extract response-to-stimuli-derived interpretable features that objectively assess patient behaviors. The cohort analysis encompasses control subjects (CTRL) and persons with Alzheimer’s disease (AD) or Mild Cognitive Impairment (MCI), Parkinson’s disease (PD), and Parkinson’s disease mimics (PDM). Overall, results suggested that the control group had significantly longer smooth pursuit distance (p < 0.05) in contrast to other cohorts. Additionally, the average hypometria prosaccade gain was significantly smaller (p < 0.05) for PD and AD/MCI relative to CTRL. The number of omitted saccades relative to a presented stimulus in the antisaccade task and latency were significantly greater (p < 0.05) for PD and AD/MCI compared to CTRL. These features, as oculographic biomarkers, can be potentially leveraged in distinguishing different types of neurodegenerative disorders in their early stages, yielding more objective and precise protocols to monitor disease progression.
I. Introduction
Neurodegenerative disorders (NDs) encompass a number of adult-onset and progressive neurological disorders that manifest with central nervous system degeneration and present with impairments of movement, coordination, mood, and/or cognition. NDs encompass a number of diseases, including Alzheimer’s Disease, Parkinson’s Disease, Frontotemporal Dementia, and a number of other related or mimicking conditions. Advancing age strongly correlates with the risk of developing ND; as a group, NDs represent a tremendous socioeconomic and personal burden, particularly with lifespan increases in many countries [1]. AD is, both, the most common ND and the most common form of dementia worldwide, followed by Vascular Dementia and Dementia with Lewy Bodies [2]. And PD is the most common neurodegenerative movement disorder that affects 2–3% of the population ≥ 65 years of age worldwide [3], [4].
Many NDs have a prodromal stage. Persons presenting with mild cognitive impairment (MCI) may, in some cases, be in prodromal stages of AD, before more fulminant memory and linguistic decline occur. The diagnosis of MCI secondary to AD is given to people with MCI who have abnormal brain positron emission tomography (PET) scans or whose spinal fluid contains increased amyloid beta protein or hyperphosphorylated tau protein. In contrast, PD is an ND defined on, primarily, clinical terms based on the presence of bradykinesia combined with either rest tremor, rigidity, or both. Notably, the clinical presentation of both AD and PD is multifaceted and includes many overlapping or non-motor symptoms [5]. Lastly, co-pathology is common [6], [7]. However, the gold standard test for a definitive diagnosis of PD or AD requires pathology evidence of neurodegeneration via autopsy. Guided by the awareness of disease subtypes, treatment plans vary from person to person, emphasizing the need for precision medicine and personalized management.
The complex and multifaceted presentation of NDs means that accurate diagnosis may require months or years. A major goal of current clinical research in NDs is improving early and accurate disease detection, which would facilitate implementing treatment as early as possible. We also need better tools to assess disease progression in this group of disorders [8]. Many approaches are being undertaken to identify biomarkers for early detection to enable accelerated neurological diagnosis. Specifically, algorithms have been developed to provide gait analysis to monitor symptoms in PD [9]. Acoustics in terms of articulatory and phonatory aspects of speech and voice are used to support automatic detection and severity assessment of PD [10].
In this work, we use non-invasive eye tracking, an experimental method of recording eye motion and gaze location across time and tasks. Eye tracking data contains not only rich information on eye movement but also provides quantitative insights into how the brain works. In a review of oculomotor features of the major age-related movement disorders, Anderson and MacAskill [11] suggested that oculomotor signs can be leveraged for differential diagnosis in NDs, such as PD [12], [13], [14], AD [15], [16], spinocerebellar ataxia [17], [18], [19], and Huntington disease [20], [21], [22]. The clinical assessment of extraocular muscle (EOM) function, at minimum, evaluates fixation and the delay or accuracy of a reflex saccade to a stimulus, along with an assessment of smooth pursuit. More complex physiological tasks, such as antisaccades, memory-guided saccades, and repetitive to-and-fro saccades, among others, are often included on a case-by-case basis [23]. Typical evaluations include an examination of the ability of individuals with NDs to fixate, track a moving target, and perform saccadic eye movements. In previous studies, AD patients were found to have delayed initiation of saccade for fixation, along with shorter fixation periods than healthy controls [24], [25]. Pavisic et al. discovered that, during smooth pursuit tasks, the AD group spent less time pursuing the target by considering the number of samples based on the ratio between the eye and the target velocity [25]. Saccadic eye movement abnormalities in AD were observed when patients had higher latency and latency variability for saccade initiation, less accurate prosaccade, and a larger number of saccades to fixate the target [26], [25], [27]. Moreover, Noiret et al. [26] showed that AD patients made more uncorrected antisaccade and had a longer latency to initiate a corrective saccade in the antisaccade task.
For PD, Antoniades and Kennard [28] suggested that the most consistent ocular motor abnormality in PD is saccadic hypometria, in which the primary saccade undershoots the target, especially with repetitive to-and-fro saccades [29], [30], [31]. There are also deficits in the initiation and performance of self-regulated tasks, including antisaccades and memoryguided saccades [30], [32], [33], [34]. Voluntary saccade execution dysfunctions along with difficulties inhibiting reflexive saccades in PD were reported in [35] when PD patients showed higher pro saccadic latency, more errors in the antisaccade task, and an increased number of disinhibitions in the delayed antisaccade task. These results are consistent with an impairment of frontal-basal ganglia circuits that leads to deficits in controlling voluntary saccade generation [36]. Additionally, repeated trials of mixed-up pro and antisaccade tasks in a block increased prosaccade and antisaccade error rates for PD patients compared to single-task blocks [37]. Moreover, Participants with PD produced more saccades during smooth pursuit than the control group, and some of them also showed impaired binocular coordination when they started the pursuit in the study by Wu et al. [38]. Tsitsi et al. [39] found that median pupil size and longest fixation period significantly differed between PD and healthy controls.
One limitation of the available literature is the assessment of a singular ND using single or few extraocular tasks. By analogy, this would be akin to attempting to diagnose a condition based on a single or few examination findings. In contrast, the strength of the current study is that the features we present were significant in distinguishing between different NDs (AD/MCI, PD, and PDM) and a control group with the analysis of multiple distinct tasks. Second, most of these studies include 20 or 30 minutes of recording on a single task instead of our protocol which favors faster, more efficient methods. Five tasks (36 trials in total) analyzed in this paper take under 12 minutes to complete in a multi-modal multi-test rapid battery. Third, in a meta-analytic review of 38 articles in the literature [14], the number of participants (mean 14.2 per study for PD and 11.8 for controls) was reported. The total population (n=155), especially the number of controls (n=59) and the number of PD (n=41), included in this study is relatively large in the topic of eye-tracking metrics for evaluating NDs, bolstering the significance of our findings. Last but not least, a manual review of eye-tracking data results in inefficiently analyzing different metrics associated with eye movement when evaluating a patient’s condition. Hence, it becomes advantageous to automatically analyze the eye movement to identify eye motion states and extract quantitative features that are interpretable, shedding light on the functioning of different parts of cognitive systems. Accompanying parameters, such as acceleration, amplitude, and duration thresholds, with velocity unlike many other approaches [40] only considering saccade detection, our algorithm shows promising results on the identification of saccade, fixation, smooth pursuit, and blink from eye-tracking data. Notably, our analysis is implemented in an automated pipeline that can accomplish analysis with minimal human supervision 1. The method is intended to assist clinicians in assessing the presence and monitoring the disease progression of different NDs. A diagram of our automated pipeline is illustrated in Fig 1.
A block diagram of the main modules in our automated pipeline
II. Materials
A. Data set
NeuroLogical Signals is an ongoing multi-modal corpus being collected by the authors of this study and currently contains 155 participants, including control subjects (CTRL), people with Alzheimer’s disease (AD) or Mild Cognitive Impairment (MCI), Parkinson’s disease (PD), and various Parkinson’s disease mimics (PDM). All ND patients were seen at Johns Hopkins Medicine, and all participants signed informed consent. This study was approved by the Johns Hopkins Medical Institutional Review Board. The AD [41], PD [42], and PDM [43], [44], [45] groups contained patients with diagnoses that were clinically established. Fourteen participants with MCI, of which six were diagnosed as MCI due to AD, were combined with six AD patients forming the AD/MCI group to show a more general behavior of dementia. All participants in the PD group had a predominant clinical syndrome of Parkinsonism. The PDM group encompassed varying degrees of PD-related movement disorders, atypical Parkinsonism, and secondary Parkinsonism, including Progressive Supranuclear Palsy [43], Dementia with Lewy Bodies [45], Corticobasal Syndrome [46], and Multiple System Atrophy [44]. In this study, all PDM participants were initially determined to have physical findings of parkinsonism and met, at some point in their work-up, “Possible Parkinson’s Disease,” but whose diagnosis evolved over subsequent visits. While inclusive of multiple etiologies, the PDM group contains persons who could be misdiagnosed with Probable Parkinson’s Disease, so finding features that could differentiate between PD and PDM is highly relevant. The table I shows the cohort study’s demographics and disease severity statistics. We report sample size, sex, age distribution, and scores on the Montreal Cognitive Assessment (MOCA) for each experimental group. In addition, we report clinical dementia rating scale sum of boxes (CDR-SB) for the AD/MCI group and unified Parkinson’s Disease Rating Scale Part III (MDS-UPDRS III) for the PD and PDM groups.
NLS Data Set
B. Data collection
The eye tracking data was recorded with EyeLink Portable Duo produced by SR Research. The setup included an eyetracking unit with a camera and an infrared torch, a host PC connected to the eye tracker and dedicated to data acquisition, and a display PC on which stimuli were presented during experiments. The display screen was 380×215 mm (1920×1080 in pixels). The distance from the participant’s head to the screen was 500 mm. We employed the head-free mode in which a sticker was positioned in the middle of the participant’s forehead to localize their eyes. This mode was preferred over head-fixed since the latter requires chin rests, which was problematic for some participants with postural and motoric problems. Moreover, a chin rest would interfere with the tasks that require the participant to speak, as described in Subsection II-C. The EyeLink system provided the coordinates of where the eyes looked on the screen (gaze) based on the detected pupil and corneal reflex. The device was set up to track both eyes with a sampling rate of 1000 Hz per eye, and we asked the participants to try to remain as stable as possible while completing the visual tasks. In all tasks, the image background was grey XXXX, and the targets were of high contrast. The screen luminance was maintained, whereas the environmental light was very similar across all the participants. Calibration and validation were performed at the beginning of the session, and drift correction was performed before each task to ensure the system was calibrated well enough. The raw signal provided by the system was composed of time sequences of [x, y] positions, velocities, and pupil size. The stimuli signal could also be extracted from eye-tracking data files to enable task-specific analysis.
C. Tasks
Five tasks, illustrated in Figure 2, were performed to assess the ocular motor and cognitive behavior in NDs:
Example sequences of screens for the five tasks employed in this work along the timeline. Prosaccade and Antisaccade are almost identical, except the participant is instructed to saccade towards or away from the target. Smooth Pursuit involves visual tracking of a target with a predetermined pattern (infinity sign in this example). Colorful text is used for passage reading in the Rainbow Passage task. Cookie Theif involves visual exploring of an image with a domestic scene.
1) Smooth Pursuit
The participants followed the target red dot with a diameter of 22 pixels, equivalent to 0.5° visual angle across the screen. Seven smooth pursuit trials with different stimulus moving patterns were performed (horizontal line: n = 2; vertical line: n = 2; infinity pattern: n = 3). Two target velocities at 0.2° /s and 0.4° /s were used horizontally with x amplitude = 14° (trial one and two) and vertically with y amplitude = 8° (trial three and four). The target red dot was moving at x velocity = 0.2° /s with x amplitude = 12° and y velocity = 0.4° /s with y amplitude = 6° in the first infinity pattern (trial five), x velocity = 0.4° /s with x amplitude = 3° and y velocity = 0.2° /s with y amplitude = 6° in the second (trial six), and x velocity = 0.1° /s with x amplitude = 18° and y velocity = 0.2° /s with y amplitude = 8° in the third (trial seven). Each trial lasted 18 s.
2) Prosaccade
Participants were initially presented with a green dot with a diameter of 0.5° visual angle in the center of the screen. After this, two target red dots of the same size were subsequently shown in opposite directions, either left or right horizontally or up or down vertically. The participants were asked to look at the target as quickly and accurately as possible when it appeared without moving their heads. The targets appeared in possible locations: ±2°, ±4°, ±6°, ±9°, ±10°, ±11°, or ±18° randomly interleaved and presented in equal or unequal numbers to opposite directions. The prosaccades task was composed of eight horizontal trials and eight vertical trials, which were mixed in order. The target locations, stimuli time, and orders were different for each trial, but constant for all participants. This means that the time a green or a red dot was on the screen varied between 900 and 2100 ms depending on the trial to avoid temporal predictability of target onset [47], but this time and the order of the trials were the same for all participants.
3) Antisaccade
Similar to the Prosaccade task, the participants were initially presented with a green point subtending at 0.5° visual angle in the middle of the screen. After this, a red point was shown either to the right or left side of the screen horizontally or to the up or downside of the screen vertically. The participants were asked to look at the opposite region of the screen. The possible target locations were ±9°, ±10°, ±14°, ±16°, or ±18° in the horizontal direction and ±4°, ±5°, ±8°, or ±9° in the vertical direction. The red point lasted variously, ranging from 900 ms to 2100 ms. The resting time between each trial also varied between 900ms to 2100ms. These settings were constant across all participants. Ten horizontal trials followed by ten vertical trials were performed in this task. The main purpose of the antisaccade task is to test disinhibition, which is defined as “the inability to withhold a prepotent response or suppress an inappropriate or unwanted behavior. It can refer to the production of socially inappropriate comments and/or actions.” [48]
4) Cookie Thief
A task of self-directed visual exploration, the participants were asked to describe the depiction within one minute [49]. No prompting was provided regarding what to describe, though subjects were encouraged to continue if they provided a short response. The picture depicts a familiar scene from everyday life with distinct characters, activities, and place contrasts [50]. Elementary keywords were expected to be used when describing the scene. Cookie Theif has been one of the most used tasks to elicit disclosure abilities and is particularly useful in assessing the integration of cognitive–linguistic abilities [50], [51].
5) Rainbow Passage
The participants read the Rainbow passage [52], in which the text was colorful, thus adding another level of complexity to the task compared to black text. There are two variations: 1. the participants will read part of the passage out loud. 2. The participants will read the rest of the passage to themselves instead.
The tasks were intended to be at an appropriate difficulty level, avoiding saturatingly large number of errors from being too difficult and limited effects of measures from being too easy [47]. The tasks utilized represent a select collection of a more comprehensive testing battery. The tasks listed were though to provide a complementary screen of cognitive dysfunction common to NDs. For instance - prosaccade (vigilance, alertness, motoric aspects of the saccade generation system), antisaccades (sustained attention, error detection, task sustainment), cookie theft (visual attention/monitoring v. inattention/neglect, spontaneous language generation, attentional shifts), and Rainbow passage (reading, dictation, articulation, phonation, etc.). We hypothesized that these tasks could produce different features indicative of motor and cognitive-related factors, which can be correlated clinically. Every task was analyzed with the eye motion states identification algorithm to find the saccades, smooth pursuits, blinks, and fixations within the recording, and a set of general eye movement features and a set of task-specific features were extracted.
III. Methods
We developed an algorithm based on distance, velocity, and acceleration thresholds to determine eye motion states from eye-tracking data automatically. Then two groups of eye movement interpretable features were extracted to evaluate the motoric and cognitive patterns in the cohorts. Finally, we studied the statistical significance of the differences between groups (AD/MCI, PD, PDM, and CTRL) employing the Kruskal-Wallis H test [53] and Benjamini–Hochberg correction [54].
A. Eye Motion States Identification Algorithm
Four eye motion states, i.e., saccade, smooth pursuit, blink, and fixation, are typically considered when analyzing eye movement. A saccade involves a rapid eye movement from one point to another, often lasting within 100 ms [55], and the velocity is commonly above 30 deg/sec [56]. Smooth pursuit involves tracking stimulus as it moves with a slow and steady eye movement. A blink involves a rapid closing and opening of the eyelid. Fixations are eye movements that stabilize the retina over a stationary object of interest [57], often lasting more than 100 ms. Because we cannot expect smooth pursuit if there are no moving targets on the screen, smooth pursuit movement is only analyzed with our algorithm in the smooth pursuit task.
The data contained time series of [x, y] gaze position on a screen (pixels), [x, y] velocity (deg/s), and pupil size for two eyes. The eye with the lowest validation error was selected for analysis. X, y accelerations were computed from x, y velocities respectively based on the equations:
where a is acceleration, Δv is the change in velocity, and Δt is the change in time. The overall velocity and acceleration were calculated by:
The goal of our algorithm is to identify the regions of eye motion states by determining the corresponding starting and ending points in the eye-tracking data. To determine in which region of the data there is a saccade, we used thresholds 40 pixels (0.91°) for saccade distance (TSD), 30 deg/s for saccade velocity (TSV), and 6000 deg/s2 for saccade acceleration (TSA). Initial saccade candidate segments were determined by finding the regions where velocity was above TSV and merged if gaps were shorter than 20 ms. For each candidate segment, the start and end points were extended while the acceleration was larger than TSA. Similar approaches have been discussed in the Eyelink Manual (Section 5.3) and [56]. After the extension, the candidate segment was identified as a saccade if the distance between the start and end was larger than TSD to remove microsaccades and artifacts during eye tracking. To determine in which region of the data there is a smooth pursuit, we used thresholds 50 pixels (1.1°) for smooth pursuit distance (TPD) and a lower-bound five deg/s in the velocity (TPV), which was intended to filter out small changes in the gaze signal. TSV was updated to 60 deg/s [58] (TSPV) to separate saccades from the background velocity of smooth pursuit eye movement. Therefore, a candidate segment was identified as smooth pursuit movement if velocity was greater than TPV and lower than TSPV during the movement and the distance between the start and end was larger than TPD. To determine in which region of the data there is a blink, the candidate segments were evaluated by checking if no eye position was detected. Blinks with gaps less than 50 ms were merged. Because closing and opening of the eyelid cause disturbance in detecting the pupil center location near the beginning and end of the blink, the algorithm was configured to fully classify the blink segments from rise to drop in the gaze position. Finally, all segments that were not saccade, smooth pursuit, or blink were labeled fixations. Data with a summed blink duration or missing gaze coordinates greater than 20 percent of the total trial length were removed during quality checking. The detailed functions are described in Algorithm 1.
Fig 3 shows several event graphs generated by the eye motion states identification algorithm for a participant and three tasks. Identified blinks were marked in yellow, saccades in red, and fixations in blue, whereas solid lines represent the gaze positions, and dashed lines represent target positions. Fig. 3 (a) shows the gaze coordinates of one eye of a participant who tried to follow the infinity pattern stimulus in the smooth pursuit task. In the prosaccade task (Fig. 3 (b), the reaction time feature, the black bar, indicates the delay to initiate a reflexive saccade after the stimulus appears. It includes an example of the participant with two hypometric saccades, each followed by one corrective saccade. The participant in Fig. 3 (c) performed a saccade in the correct, contralateral direction during the antisaccade task after an impulsive/erroneous prosaccade initially. Stepped saccades and occasional fixations are shown in (d) for the Rainbow Passage task. In (e), we can see the identified saccades and fixations in image exploration for the Cookie Thief task. The event graphs of all the tasks and participants were visually checked to validate the output of the algorithm.
Visualization of identified eye motion states. (a), (b), (c), and (d) are plots of eye movement position vs. time. For a better view, the Cookie Thief task is visualized with eye movement position x vs. position y in (e).
B. Feature Analysis
Two sets of interpretable features, listed in Table II, were extracted. One was only based on eye movement with respect to saccade, smooth pursuit, blink, fixation, and relative pupil size change, without requiring the information of stimulus to compute. The other set of features depended on the interaction with stimuli, i.e., requiring stimulus position to calculate, specifically for smooth pursuit, pro, and antisaccade tasks. According to the internationally standardized prosaccade and antisaccade protocol by Antoniades et al., [47], we measured latency in saccade initiation, the gain of the prosaccade, hypometria or hypermetria percentage, correctness (error saccade or no response), and peak saccadic velocity. In addition, we measured the variability of the eye movement by the number of saccadic adjustments after the first saccade and the change in the non-dominant direction in which the stimulus was moving. These sets of eye-tracking metrics could reveal the motoric patterns in different NDs in different tasks and also cognitive patterns in the antisaccade task, as it was cognitively demanding to suppress the reflexive saccades.
Summary of the extracted general eye-movement and task-specific features. The description for each feature was reported. Abbreviations: PS, prosaccade, AS, antisaccade, #, number
1) General Eye Movement Features
After saccades, smooth pursuits, blinks, and fixations were identified, the features of each eye motion state, including segment duration, distance, and velocity (average and maximum), were extracted. Then, for each trial and participant, all features were characterized by count, mean, maximum, median, and standard deviation statistics. For instance, the ideal case in a smooth pursuit trial would constitute only one complete segment of smooth pursuit movement from start to end, but it is also likely that a participant may have a certain count of smooth pursuit segments, which might be caused by intrusive or catch-up saccades. Similarly, as an example, in the Rainbow Passage task, a participant will have a certain number of fixations and saccades, which will also have a mean and max velocity. Because Pro and Antisaccade tasks had multiple similar trials based on the direction target was moving, i.e., horizontal or vertical, each of them was further split into two groups.
The count, mean, maximum, median, and standard deviation statistics of these features were also performed across the trials at the group level. We hypothesized that the subjects with NDs might not perform consistently for every trial in the groups. Lastly, since the pupil area information in our data had arbitrary units not calibrated across patients according to the Eyelink manual, we calculated and analyzed the unitindependent relative percentage change in pupil size between subsequent points instead of absolute values. The equation is as follows:
where xt is the pupil size at every timestamp t. The measurements were noise-limited, with noise levels of 0.2% of the diameter.
2) Task-specific Features
The task-specific feature extraction algorithm was employed in the smooth pursuit, prosaccade, and antisaccade tasks. To quantify a participant’s response to stimuli, the features extracted from the smooth pursuit task were:
Difference between the gaze and target positions. The smaller the difference is, the more accurate tracking of the stimulus.
Pursuit gain, ratio between the eye and the target velocity. Saccades and blinks were excluded, and only a ratio greater than 0.5 was considered pursuit gain. The cutoff was applied to dismiss eye movements after saccades and turnaround points in each trial [25].
In prosaccade and antisaccade tasks, developing upon the recommended outcome measures from Antoniades et al. [47], we measured a new set of features, specifically:
Latency of the correct prosaccade or antisaccade, such as the response time from the appearance of the target to the start of the saccade. We hypothesized that NDs would demonstrate a greater latency to initiate a pro- or antisaccade after stimuli presentation than controls.
Correctness, whether saccade in the correct direction or errors of omissions (no response at all). For antisaccades, participants were supposed to look away from the stimulus instead of looking at it, like in the prosaccade task. Therefore, the number of saccades in the wrong direction (error saccades) was also derived. We hypothesized that the antisaccade task would be useful to differentiate the AD/MCI and the CTRL groups due to the added cognitive loading to inhibit a prepotent, i.e., prosaccade towards stimuli [47].
Gain of the prosaccade, the ratio of actual saccade amplitude to target amplitude. We calculated the minimum, maximum, and standard deviation of the gain. We further defined that the prosaccade is a hypometria saccade if the prosaccade undershoot the target position by at least 1.5° or a hypermetria saccade if overshoot by at least 1.5° [25]. Based on this separation, the mean hypometria gain and hypermetria gain were extracted. We also calculated hypometria saccade or hypermetria saccade frequency by the percentage of their respective count over the total prosaccade count, and the accuracy of prosaccade was found by 1 − hypometria percentage − hypermetria percentage, which means that the attempted prosaccade falls within the 1.5° range of the target position. Since we did not ask the participants to make “mirror” saccades in the antisaccade task, i.e. to the exact opposite location on the display, the gain of antisaccade therefore was not measured.
Variability. The number of saccades after the intentional saccade initiated until the stimuli disappeared was measured. This feature included corrective saccades If the participant did not reach the target accurately at the initial attempt and the number of saccades when fixating on the target. Moreover, since the stimulus moved horizontally or vertically, the position in the other non-dominant direction stayed the same. Visually as time passed, the position would form a straight line as the dashed magenta line in Fig. 3 (b). We calculated the deviations, marked as the double arrow in the graph, to this position line. A small change in x generates a small pulse in y, and it shouldn’t. When the target moves horizontally, we expect that the participants saccade to the target at the same level vertically in the prosaccade task: the vertical deviation should be as small as possible.
Peak saccadic velocity and saccade duration of the pro and antisaccade.
IV. Results and Discussion
We applied our eye motion state identification algorithm and feature extraction pipeline introduced in Section III. The results are summarized in Table III and explained in the sections below. We also indicate the number of subjects per task in this table, as this number is not constant across tasks due to errors or noise during data collection. For each pairwise feature, we report the corresponding p-values based on H-statistic [53] to determine if there were any statistically significant differences between the experimental groups. The Kruskal-Wallis H test is a nonparametric test whose null hypothesis is that the mean ranks of the groups are the same. To control the false discovery rate (FDR) in many features, we applied Benjamini–Hochberg correction to each pair-wise comparison [54]. The error rate, α, was set to 0.05. We also report the area under the ROC curve (AUROC), which can be used as a criterion to measure the feature’s discriminative ability.
Pairwise Kruskal–Wallis H test results for statistically significant features (p < 0.05) from the eye tracking analysis
A. General Eye Movement Features
In the results section, we plot distributions of all groups in boxplots indicated by different colors as in Fig 4. The results of the smooth pursuit task show that the control group had a significantly greater (p < 0.05) average distance for the smooth pursuit movement than all other groups, which means that they were less interrupted by other eye movements. The AD/MCI and PD groups also exhibited significantly longer (p < 0.05) average saccade duration than the control group. The standard deviation of saccade distance was greater (p < 0.05) for the AD/MCI group compared to the control group. Both AD/MCI and PD groups had significantly higher (p < 0.05) saccade velocity than the control group. Overall, participants with Alzheimer’s appeared to have more significant deficiencies in smooth pursuit performance. Our experimental results seem to support the findings of visual tracking impairments in Alzheimer patients [59]. In Cookie Thief (visual exploration) task, the PD group showed significantly fewer fixation counts and a higher variability (standard deviation of fixation duration and saccade duration) (p < 0.05) than the control group in Fig 4 and 5. The PDM group differed significantly from the PD group in the standard deviation of saccade duration as well. Moreover, in the reading-out-loud trial of Rainbow Passage, there were no significant differences between people with AD and other groups in terms of fixation count, saccade count, and standard deviation of fixation duration. However, in the silent version, the AD/MCI group had significantly (p < 0.05) more fixations and saccades and a higher standard deviation of fixation duration than the control and PD groups. Compared to reading out loud, groups except for AD had notably fewer saccades and fixations while reading silently. It seems that loud reading made it more difficult for the CTRL, PD, and PDM groups due to the multi-task nature of additional speaking that involves decoding, articulation, and prosody [60], but it had a different effect on the AD group. This could be because reading and listening to themselves helped the participants with AD visually scan and understand information from the passage. The average fixation duration was significantly larger (p < 0.05) for PD and PDM groups compared to the control group in both trials of Rainbow Passage. 5. Lastly, the relative change in pupil size was found to be significantly smaller (p<0.05) for the PDM group compared to the control, AD/MCI, and PD groups in both Cookie Thief and Rainbow Passage tasks. In work [61], pupillary size served as a measure of different attentional demands in local vs. global patterns. Presumably, cognitively demanding tasks (CookieThief, ReadRainbowPassage) require more sustained attention than simple reflexive saccades or antisaccades.
More significant plots are included in the appendix. The center line corresponds to the median. The data space is divided further recursively, and more quantiles are added. The whisker extends to the most extreme data points where the letter values are reliable estimates of their corresponding quantiles, and the outliers are represented with the “diamond” symbol. “Asterisks” indicate significant differences: ∗p < 0.05, ∗∗p < 0.01, ∗∗∗p < 0.001.
The center line corresponds to the median. The data space is divided further recursively, and more quantiles are added. The whisker extends to the most extreme data points where the letter values are reliable estimates of their corresponding quantiles, and the outliers are represented with the “diamond” symbol. “Asterisks” indicate significant differences: ∗p < 0.05, ∗∗p < 0.01, ∗∗∗p < 0.001.
B. Task-specific Features
In Fig 5, the PD group had a significantly larger (p<0.05) average difference between the gaze and target positions than the CTRL group in Smooth Pursuit trials two and four, in which targets were moving at a faster speed horizontally and vertically in straight lines. In comparison, no statistically significant differences were observed in trials one and three with a slower target moving speed. The average pursuit gain was also significantly greater (p<0.05) for the PD group compared to other groups. In Smooth Pursuit trials five and seven (infinity patterns), the maximum difference between the gaze and target positions was significantly larger (p<0.05) for the AD group relative to the CTRL group. This feature was also larger (p=0.074) for the AD group in trial six (infinity pattern), but it did not reach the statistical significance threshold, which may be due to the smaller x and y target amplitudes than the other two trials.
According to Fig 4, we found that even though all groups processed a similar frequency of generating a hypometria saccade in terms of percentage in the horizontal direction, the average hypometria gain was significantly smaller (p<0.05) for the PD group than CTRL, which means that people with Parkinson’s had trouble reaching the stimuli accurately at the first attempt. In the vertical prosaccade task, both the hypometria percentage and average gain reached the significance level (p<0.05) for AD and PD groups compared to CTRL. The PD group continued to show impaired prosaccade performance with minimum gain and hypometria gain to a more significant degree. These findings could be attributed to a consistent ocular motor abnormality in PD - the saccadic hypometria in which the primary saccade undershoots the target [29]. Also, persons with PD often demonstrate a partially reversible saccadic hypometria which is, in part, dependent upon their L-DOPA state: persons in the optimal L-DOPA ON state may perform differently/better on this task v. L-DOPA OFF state [62], [63]. In this study, we are considering the group as a whole, hence might be more likely to distinguish than AD or PDM. Similarly, the accuracy for prosaccade and latency in the saccade initiation did not differ significantly between groups for horizontal prosaccades. However, the PD groups exhibited significantly less accuracy and maximum latency (p<0.05) for vertical prosaccades than the control group. Moreover, the frequency of hypometria in terms of percentage became significantly higher (p<0.05) for the AD/MCI and PD groups compared to the CTRL group in the vertical prosaccade task.
For the horizontal prosaccade task, the AD/MCI and PD groups showed a significantly smaller minimum gain and larger standard deviation of gain (p<0.05) than the control group in Fig 5. Therefore, both groups had a lower magnitude of the shortest hypometria saccade and higher variability of saccade gain. The number of saccades after they initiate towards the target until the stimuli disappear is also significantly higher (p<0.05) for the PD group than the control group according to 5, exhibiting more corrective saccades and less stability in fixation. In addition, the deviation of the gaze position in the non-dominant direction significantly differed (p<0.05) in CTRL and AD/MCI vs. PD group. It is possible that people with Parkinson’s had less control in the less sensible direction while focusing on the main task due to motoric impairments.
For horizontal antisaccades, there was no significant difference in the number of saccades in the wrong direction - prosaccade, between groups. However, for vertical antisaccades, the results showed that the PD made more wrong prosaccades relative to the CTRL group (p<0.05) in Fig. 4. Also, the AD/MCI, PD, and PDM groups had significantly more errors of omission, the number of omitted saccades relative to a presented stimulus, than the control group (p<0.05) for vertical antisaccades. This scenario means that even if the controls made saccades in the wrong direction, they eventually would make a correct antisaccade, while it did not hold true for people with NDs, suggesting impairments in detecting an error was made, formulating a correction and executing it. AD/MCI due to cognitive inattention or bradyphrenia might contribute to responding promptly. A motoric impairment in antisaccade relative to disease severity or L-DOPA state might explain this finding for the PD group. PDM is a mixed cohort with a number of EOM findings which might lead to a reduced detection of positive pathology. We posit that impairments in Cortico-Striatal-Thalamic loop circuits, either subcortically or in the neocortex, may subserve these impairments. There was also a significant difference (p<0.05) in the maximum latency of the correct antisaccade for the AD/MCI group and the average latency of the correct antisaccade for the PD group compared to CTRL. Therefore, the antisaccade task validates itself as a useful tool for measuring the executive function to suppress the reflexive saccade to a peripheral target and endogenously saccade to an equal opposite direction [28].
V. Conclusions and Future work
In this work, we collected eye-tracking data from multiple tasks to assess the motoric and cognitive impairments in different types of NDs. A major strength of our multi-modal multi-test rapid battery is portable, and the tasks are fast to complete in contrast to the previously reported versions. We employed signal processing techniques and statistical analysis to design an algorithm for analyzing eye movement and configure a wide range of features that could be used as biomarkers to assist clinicians in diagnosing NDs. First, our algorithm automatically identifies saccade, fixation, smooth pursuit, and blink eye movements, and it would potentially work with any eye tracking data as long as the position column is present with appropriate thresholds. Moreover, while maintaining interpretability and objectivity, our two sets of features represent valuable metrics to quantify the performance of different visual tasks. The set of general eye movement features provides the basic characteristics of eye movement that can be applied to any task. We found that the control group had a significantly greater average smooth pursuit distance than groups with NDs, and the AD/MCI and PD groups were characterized by having longer saccades in the smooth pursuit task than the control group (p < 0.05). In the Rainbow Passage task, the AD/MCI group made significantly more (p < 0.05) fixations and saccades than the CTRL and PD groups. In the Cookie Thief task, the PD group had a significantly larger (p < 0.05) standard deviation in saccade duration than the CTRL and PDM groups. The task-specific features encode how participants react to the stimuli in specific tasks. The PD group showed a significantly smaller average hypometria gain and accuracy (p < 0.05) in the prosaccade task. Significantly more omitted saccades are reported for the groups with NDs compared to CTRL in the antisaccade task. The utility of a PDM group was not as an intrinsic/valid construct on its’ own merits but served as the important clinical impact in relation to Parkinson’s Disease. As the multimodal corpus is an ongoing process, we are in the process of expanding our subjects in each category to balance the experimental groups regarding age, gender, and the number of subjects. By doing so, we expect our approach’s accuracy and generalizability will increase when analyzing a higher number of disorders included in our data set. In conclusion, this research aims to develop novel machine learning paradigms based on these features to improve differential diagnosis between NDs and monitor disease progression in time.
Data Availability
The authors plan to share all data produced in the present study in the future.
Appendix
Footnotes
↵1 The code, including the eye-tracker experiment configuration, is available upon publication.
References
- [1].↵
- [2].↵
- [3].↵
- [4].↵
- [5].↵
- [6].↵
- [7].↵
- [8].↵
- [9].↵
- [10].↵
- [11].↵
- [12].↵
- [13].↵
- [14].↵
- [15].↵
- [16].↵
- [17].↵
- [18].↵
- [19].↵
- [20].↵
- [21].↵
- [22].↵
- [23].↵
- [24].↵
- [25].↵
- [26].↵
- [27].↵
- [28].↵
- [29].↵
- [30].↵
- [31].↵
- [32].↵
- [33].↵
- [34].↵
- [35].↵
- [36].↵
- [37].↵
- [38].↵
- [39].↵
- [40].↵
- [41].↵
- [42].↵
- [43].↵
- [44].↵
- [45].↵
- [46].↵
- [47].↵
- [48].↵
- [49].↵
- [50].↵
- [51].↵
- [52].↵
- [53].↵
- [54].↵
- [55].↵
- [56].↵
- [57].↵
- [58].↵
- [59].↵
- [60].↵
- [61].↵
- [62].↵
- [63].↵
