Performance-based and Observational Assessments in Clinical Trials Across the Alzheimer’s Disease Spectrum

| February 1, 2017 | 0 Comments

by Philip D. Harvey, PhD; Stephanie Cosentino, PhD; Rosie Curiel, PhD; Terry E. Goldberg, PhD; Jeffrey Kaye, MD; David Loewenstein, PhD; Daniel Marson, PhD, JD; David Salmon, PhD; Keith Wesnes, PhD; and Holly Posner, MD
Drs. Harvey, Curiel, and Loewenstein are with the University of Miami Leonard Miller School of Medicine, Department of Psychiatry and Behavioral Sciences, Miami, Florida; Dr. Cosentino is with Columbia University College of Physicians and Surgeons, New York, New York; Dr. Goldberg is with North Shore Hofstra School of Medicine, Hempstead, New York; Dr. Kaye is with University of Alabama, Birmingham, Alabama; Dr. Marson is with University of Oregon, Portland, Oregon; Dr. Salmon is with USCD Medical Center, La Jolla, California; Dr. Wesnes is with Wesnes Cognition Ltd., Streatley on Thames, United Kingdom, and Department of Psychology, Northumbria University, Newcastle, United Kingdom; and Dr. Posner is with Global Product Development, Neuroscience & Pain, Pfizer, Inc., New York, New York.

Innov Clin Neurosci. 2017;14(1–2):30–39

Funding: No funding was received for the preparation of this article.

Financial disclosures: Drs. Harvey, Cosentino,?Curiel, Loewenstein, Goldberg, Kaye, Marson, Salmon, and Wesnes have no conflicts of interest relevant to the content of this article. Dr. Posner is an employee of Global Product Development, Neuroscience & Pain, Pfizer, Inc., New York, NY, USA (the work and time that went into this article were not done as part of Pfizer responsibilities). The nonprofit organization, International Society for CNS Clinical Trials (ISCTM), paid travel fees for the non-industry authors to attend ISCTM meetings that led to this paper.

Key words: Early Alzheimer’s disease, mild cognitive impairment, MCI, clinical trials, cognition, functional assessment

Abstract: Assessment of the earlier stages of Alzheimer’s disease requires different strategies than those previously developed for fully syndromal Alzheimer’s disease . This challenge is further magnified in very early stages, where symptomatology may be minimal and functional deficits very subtle to absent. This paper reviews strategies for performance-based assessment of the early stages of Alzheimer’s disease , including assessments of cognition, functional capacity, and social cognition. Meetings with an International Society for CNS Clinical Trials and Methodology working group served as the basis for this paper and its companion. The current state of the art of detection and staging-oriented assessments is presented, and information is provided regarding the practicality and validity of these approaches, with a special focus on their usefulness in clinical trials for new medication development.


Cognitive and functional decline are the hallmark signs of Alzheimer’s disease (AD) and related conditions. In fully syndromal AD, the detection of memory and functional deficits can be accomplished with clinical assessments not requiring formal testing, as these impairments are often quite obvious. In contrast, there are multiple challenges in assessing AD in the early stages of the illness, particularly when an individual has a high level of cognitive reserve and compensatory abilities. As described by Posner et al,[1] developing treatments are now targeting earlier stages of the illness, and mechanisms are being explored to prevent the development of AD in individuals who are at risk. It has become increasingly recognized that newly developed treatments eventually approved by the United States Food and Drug Administration (FDA) will need to employ outcome measures that translate into real world impact (i.e., how a person feels, functions, or survives). Since newer interventions will target the early stages of disease, there has been a focus on developing novel tests to detect the earliest manifestations of AD. This is because traditional neuropsychological tests were designed to detect changes associated with dementing conditions and are 1) not sensitive enough to identify subtle cognitive impairment present in preclinical disease states, and 2) not designed to reliably assess meaningful change over time. Similarly, commonly employed measures of functional abilities are typically inventories completed by patients and their informants. Although there is value in this approach, subjective judgments made by both patients and informants may be prone to error and response biases.

In this paper, we describe the state of the art of objective assessment of cognitive and functional impairment in AD across its early stages. We particularly focus on the following three domains:

  1. Tests sensitive to early cognitive impairments
  2. Performance-based assessment that provides standardized, objective, and highly portable means to assess deficits in functional capacity
  3. Direct observation measures, which are now a more broadly viable strategy due to technological advances, including video observation and ecological momentary assessment with smartphone technology.

What are the “early stages” of AD?

There are several different populations who are treatment targets who do not meet full criteria for AD. These include individuals who meet current diagnostic criteria for mild cognitive impairment (MCI), the diagnosis of which is agreed upon to an extent. Further, there are individuals who have subtle cognitive changes that are not substantial enough to meet criteria for MCI; in some studies these individuals are designated as “pre-MCI.”[2] Both MCI and pre-MCI could be viewed as conditions that are “prodromal AD” conditions, in that the risk for AD in each of these populations is empirically elevated compared to individuals without evidence of cognitive changes.[3]

A final group, which can be referred to as “pre-clinical,” is a group that is identified on the basis of risk factors, including either genetic liability to AD or the presence of biomarker changes that suggest increased risk.[4] At the current time, the pre-clinical group is marked by the appearance of cognitive normality,[4] but cognitive changes can be detected in these individuals. The question is whether more sensitive tests could identify earlier impairments in some or all preclinical individuals.

Limitations of the Current State of the Art

Current assessment methods used in clinical trials targeted toward cognitive and neurodegenerative disorders are limited for use in early phases of AD. In general, this is directly related to the inherent nature of the features of change that are being tracked—slow decline possibly punctuated with acute, unpredictable events—for which existing tools and methods of assessment are not optimally designed. In particular, there are two contemporary major constraints. First, clinical assessments are conducted in person at distinct spaced intervals, weeks or months apart, at the convenience of the investigator. These assessments often rely on recall of events from people whose memory may be impaired and informants whose judgments may be prone to error or memory impairments of their own. In addition, contextual aspects of a participant’s daily life (e.g., sleep hygiene, medications, pain level) that might affect true real-world performance are not captured, and as a result, data collected are at best brief snapshots that do not fully reflect real-world situations. Second, in treatment studies, there is often only a modest expected difference among cognitive and/or functional test results between placebo and treatment participants, as the degree of cognitive decline prior to and during treatment is often subtle and not dramatic in early stages of AD.

Overall Goals of Cognitive and Functional Assessment in Early Phases of AD

In AD prevention research, targeted at pre-clinical and pre-MCI populations, outcomes assessments should be capable of, at minimum, the following: 1) bi-directional sensitivity, 2) longitudinal tracking (sensitivity to change), and 3) sensitivity to impairment. Bi-directional sensitivity is required, because some successful interventions could eliminate progression to more advanced clinical states, relative to placebo-treated participants. Thus, measures must have the ability to detect both stability (due to a successful intervention) and worsening (due to treatment failures) in clinical trial participants who might have considerable variation in their baseline levels of performance. A critical goal for measures is to be sensitive to long-term cognitive and functional stability, which is the goal of prevention interventions. These interventions may not improve functioning from an apparently unimpaired baseline, but may prevent progression to MCI or AD. Beyond the basic requirements of test-retest stability of performance,[5] improvements in performance with testing might serve to mask subtle declines in placebo treated participants in preclinical stages,[6,7] thus rendering a true outcome of stability in the successfully treated pre-clinical individuals difficult to separate from retesting effects.

A final goal is to be able to detect developing impairments in cognition and functioning in preclinical, pre-MCI, and MCI populations. There are two separate issues that bear on these goals. One is the level of difficulty of the assessments. Some skills that are completely intact in preclinical cases may be impaired in MCI and fully deteriorated in cases with mild-to-moderate AD. For instance, assessment of delayed recall and delayed recognition from multi-trial list learning tests with 10 items (e.g., Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) neuropsychology battery)[8] may be extremely sensitive to the detection of early-stage AD;[9] however, these learning tests, while optimal in their difficulty level for MCI and mild AD populations, may be too easy for preclinical and pre-MCI populations and too challenging for moderate AD.

A second question regarding sensitivity to decline is whether the same aspects of cognitive and functional impairment are even relevant in preclinical and pre-MCI states as in AD. It may be that different assessment paradigms must be used to detect cognitive or functional changes early in the course of MCI. There are elements of memory impairment, discussed below, that define pre-MCI and could provide predictive utility for identifying cases at risk for progression to more severe impairments.

Similarly, performance-based functional assessments, if feasible at all, will need to be aimed at very subtle aspects of impairment in preclinical and pre-MCI cases. An example could be measuring highly complex instrumental activities of daily living (ADLs) such as internet-based banking or bill paying. Incipient signs of these deficits might not be visible to observers or even accessible to the awareness of the participant. As suggested below, ecologically valid direct observation strategies may also yield information that might not be easily observable, particularly in a cross-sectional context. In contrast, in mild-to-moderate AD, even basic ADLs may be performed only with difficulty and be readily detected even by untrained or themselves impaired observers.

Sensitivity to Impairment and Sensitivity to Change

If a cognitive or functional assessment is not sufficiently sensitive to detect impairment, it will not be sensitive to improvement with treatment. A caveat is that the impairments detected must be related to the incipient illness and not lifelong premorbid limitations in functioning.[10] In the case of the minimal impairments associated with pre-MCI, detection of impairments with historical measures such as the ADAS-COG is challenging due to ceiling effects,[11] and detection of improvements with such measures may not be possible. If preclinical study participants enter a trial performing at a level equivalent to their lifelong level of functioning on the cognitive endpoints, as expected in contemporary prevention trials, then detection of impairment is not important. Rather, the utility of the assessment measures employed will be indexed in terms of longitudinal sensitivity to decline on the part of untreated (i.e., placebo) participants and separation of this subtle decline from stabilized performance induced by successful preventative treatment. This is an extremely high bar to meet.

In contrast to studies in patients with mild-to-moderate AD, cognitive and functional impairments in preclinical and pre-MCI individuals will be minimal to absent. Thus, there is less need for concern about tasks being too challenging and leading to floor effects. There will also be fewer challenges associated with participants not understanding instructions or having gross linguistic, perceptual, or praxic deficits, leading to findings of nonspecific impairment. However, the relatively unimpaired status of the participants at entry to the clinical treatment trial produces several challenges. The challenges are reduced in inverse proportion to the level of impairment seen at entry into the study. A clear challenge for assessment in prevention trials will be balancing minimal practice effects and the sensitivity of measures. Alternate forms of neuropsychological tests are often challenged by form-to-form variance that is greater than differences associated with practice effects.[12]
Potential assessment instruments to identify effective prevention interventions will need to meet several criteria, such as the following:

  1. Minimal and quantifiable improvements in performance associated with retesting alone (i.e., practice effects)
  2. Sensitivity to the subtle declines seen in placebo-treated patients
  3. Ability to be repeated multiple times throughout a lengthy trial.

Further, assessment of functioning, including performance-based tasks, could also be included as outcomes pending regulatory decisions and development of sufficiently sensitive measures.

Advances in Assessment of Early Cognitive Impairment

Cognitive deficits in MCI and in AD have been widely studied. For treatment studies of AD, the ADAS-Cog is a widely used outcome measure. The ADAS-Cog, although used in some studies of MCI, is not as suited to this population because many of the cognitive impairments examined in the ADAS-Cog are not yet present in MCI, thus making most of the items uninformative. Loewenstein et al[13] have found that older adults with pre-MCI have a 6-fold to 8-fold risk of progressing to MCI or dementia relative to elderly cognitively normal individuals over a two to three year period.[2,13] Thus, the question arises: When one is attempting to identify important deficits in pre-MCI or preclinical individuals, is a more challenging version of the tasks used to assess deficits in MCI required or are the deficits different, requiring different tests?

The general focus in assessment of the earliest signs of AD has been on the detection of episodic memory impairments, both in terms of impairments in learning/encoding and rate of forgetting. Increasing difficulty in standard tasks can be accomplished by adding more stimuli to the assessment (e.g., a longer story), modifying the encoding potential of the stimuli, changing the presentation rate, and altering the recall/recognition instructions. However, increasing the difficulty of tests has some risks. These include questions as to whether more difficult tests measure the same construct as the easier tests and whether there are possible reductions in the test’s psychometric properties such as floor and ceiling effects or test-retest reliability induced by increasing difficulty. More difficult cognitive tests may be more highly related to global intelligence and therefore associated with reduced performance in healthy, but less educated adults. The closer the score is to the floor of the test, the greater the risk for reductions in test-retest reliability due to regression to the mean and general instability of scores at the extremes of distributions.

Shortcomings of existing measures and targets for modification in commonly used memory measures for early AD include the following: A) they focus on passive rather than active encoding strategies, B) they are susceptible to attentional issues, C) they are influenced by level of education and cognitive reserve, D) they do not address vulnerabilities of AD patients to semantic interference,[14–16] and E) they do not dissociate between memory deficits and deficits in utilizing semantic cues.[17–18] Commonly utilized tests typically focus on retrospective memory as opposed to prospective memory (remembering to remember intended actions), a type of memory increasingly found to be implicated in MCI.[20] Measures of prospective memory may be as sensitive, or more sensitive than, traditional measures of retrospective memory in MCI patients.[21,22]

An alternative possibility for prevention trials is the deployment of novel tests to select samples in preclinical states. To this end, there has been a focus on the development of neuropsychological tests that tap into abilities such as 1) associative memory necessary for “binding” representations of two or more stimuli, 2) pattern separation necessary to distinguish between two similar memory representations, and 3) prospective memory required to remember a delayed intention to act at a certain time in the future. At least two of these types of tests (face-name associative memory and pattern separation) have been shown to be related to AD imaging biomarkers (Pittsburgh Compound B [PiB] imaging and functional magnetic resonance imaging [fMRI], respectively) and are being incorporated as secondary measures into the A4 prevention trial.[23] More recent developments in biomarker imaging data suggests that beta amyloid deposition may have a predilection for areas within the anterior and posterior cingulate, precuneus, and selected regions of the frontal, temporal, and parietal lobes, which can be visualized earlier than medial temporal lobe atrophy. This may give rise to subtle dysfunction in specific aspects of memory, efficiency of cognitive processing, and executive function that may not be detectable with typical clinical trials outcome measures.

Loewenstein et al[24] recently developed several measures sensitive to impairments in otherwise unimpaired preclinical individuals. One of these, the Loewenstein-Acevedo Scales of Semantic Interference and Learning (LASSI-L[24]) has been shown to have good psychometric properties and robust sensitivity and specificity in differentiating normal controls from preclinical states identified with MRI. The LASSI-L has high re-test reliability in more impaired patients using National Institute on Aging–Alzheimer’s Association (NIA-AA) criteria for MCI due to AD (MCI-AD[24]). Similar findings were evident for mildly demented AD patients.[25,26] Discriminative validity studies were conducted to distinguish between 34 MCI and 47 cognitively normal subjects using step-wise logistic regression, yielding a sensitivity of 87.9 percent, a specificity of 91.5 percent, and an overall correct classification rate of 90.0 percent. This far exceeded classification rates from traditional neuropsychological measures, such as delayed recall of passages and category fluency.[26] The LASSI-L also correlated with MRI measures of medial temporal atrophy and with high amyloid load in the precuneus and posterior cingulate and temporal regions.[24] Recent findings by Loewenstein et al[26] have shown that the inability to recover from the effects of proactive interference is highly predictive of regional and total amyloid load among community-dwelling elders with no clinical diagnosis of MCI and normal scores on traditional neuropsychological measures.

Similarly, Parra et al[27,28] have developed a short-term visual memory binding test (SVMB) that incorporates a paradigm measuring the ability to determine a change in features, such as shape or color, or changes in shape-color binding. They have found that the SVMB test was able to identify impairments among asymptomatic carriers of the E280A single presenilin-1 mutation, which causes autosomal dominant AD.[28] Moreover, the SVMB test is sensitive to early AD,[27] and can distinguish between early AD and other neurodegenerative conditions, as well as between AD and depression.[27–29] A relationship between SVMB test performance and early changes in the hippocampus and surrounding medial temporal regions has been described.[30] Moreover, because the shape–color combinations in the SVMB test are quickly overwritten in short-term visual memory, they are not susceptible to practice effects. Given that the SVMB test has been shown to be able to detect memory-binding deficits in asymptomatic carriers of Presenilin 1 mutations, it may be sensitive to memory changes among preclinical subjects who have elevated amyloid load. Finally, the use of non-verbal information and reliance only on change detection makes the SVMB test promising for studies in multicultural groups.

Assessment of Functional Skills and Outcomes

There is recent evidence to suggest that subtle but measureable and detectable functional impairment may occur in the early phases of AD. The traditional clinical view in the Alzheimer’s dementia field has been that functional impairment first emerges in the formal dementia stage. However, the intriguing proposition that detectable functional change actually commences much earlier in the AD disease process, in MCI, and possibly as early as the preclinical stage, is receiving increasing attention.[31–34]

Prior research by several groups[35–39] has shown that complex functional skills (independent activities of daily life, or IADLs) show impairment in patients with MCI and continue to decline over time.[36] In particular, financial capacity is a higher order functional skill that is highly sensitive to MCI and mild AD.[36,37,39] The vulnerability of financial capacity in MCI and AD raises the possibility that measurable financial decline may also occur and is detectable in persons with preclinical AD.[34]

Performance-based measures have been found to be sensitive to treatment effects for both pharmacological and rehabilitation interventions in severe mental illness. Further, several studies have shown that performance-based functional capacity assessment detects fairly substantial deficits in individuals with MCI and that the performance of these populations separates from both more severe AD on the one hand and healthy older individuals on the other. Performance-based functional assessment, despite being an alternative form of cognitive assessment, is appealing because of its immediate clinical relevance as a direct measure of functioning and not a distal measure such as a word list recall. The FDA has endorsed the use of these measures as a functional co-primary in treatment trials aimed at cognition in severe mental illness and has verbally expressed interest in these in early AD as well.

A critical factor here is the sensitivity of the functional assessment measure employed. Detection of functional impairment in cognitively normal individuals with a biomarker for AD will require instruments sensitive to very subtle functional changes. Global informant report rating measures commonly used to characterize functional decline in late MCI and AD type dementia are unlikely to detect presumably very subtle functional changes in cognitively normal persons with an amyloid biomarker.[34] In order to maximize sensitivity, such new assessment measures should incorporate the following features:[34]

  1. Assess cognitively complex functional abilities relevant to independent living and sensitive to early decline
  2. Assess functional ability using an interval-scaled, direct-performance measure that evaluates performance variables in a highly detailed and granular manner
  3. Include time limitations for performance items in order to enhance the item’s cognitive complexity without changing basic task demands
  4. In addition to performance items, include completion time variables in order to capture subtle processing speed changes.

As noted above, financial capacity specifically represents an IADL critical to independent living. In this vein, the University of Alabama at Birmingham group has recently developed a new functional assessment measure, the Financial Capacity Instrument—Short Form (FCI-SF[40]), which evaluates performance on tasks of monetary calculation, financial conceptual knowledge, use of a checkbook/register, and use of a bank statement and also includes time to completion variables. The FCI-SF takes 15 minutes or less to administer to cognitively normal older adults, and fulfills the four measure characteristics above for functional assessment measures in preclinical AD. A fuller description of the development and psychometric characteristics of the FCI-SF can be found elsewhere.[40–42] In promising initial field testing, the performance and timing variables of the FCI-SF have proven sensitive to early financial skill declines in cognitively normal older adults in the Mayo Clinic Study of Aging who are amyloid positive based on C11 PiB positron emission tomography (PET) neuroimaging.[40–42]

Similar promise may hold for broader assessments like the UCSD Performance-Based Skills Assessment.[43] Several studies using the UPSA have shown that performance on tests that measure the ability to perform everyday functional skills are impaired in patients with MCI compared to healthy controls, with MCI patients performing better than patients with AD. Goldberg et al[44] used the UPSA to compare healthy individuals, people with amnestic MCI, and AD patients and found that patients with amnestic MCI had substantial effect size (d=0.86) impairments in UPSA performance compared to healthy individuals. These MCI patients were not rated as manifesting any impairment on the ADCS-ADL scale compared to healthy controls (HC), while both groups were less impaired than the AD patients. In the sample as a whole (MCI, AD, HC) the correlation between the ADCS-ADL subscale and the UPSA was substantial (Spearman rho=0.63). Thus, the UPSA was sensitive to impairments in MCI patients not detected by the ADCS-ADL scale and manifested considerable evidence of validly identifying the correlates of real-world functional deficits. Further, an abbreviated version of the UPSA was developed for use in MCI,[45] which also was found to have excellent discriminant validity and a very brief (7–10 minute) administration time. Given the correlations previously published between UPSA scores and cognitive test performance in healthy controls,[46] it would not be surprising to see impaired functional or performance-based skills in individuals whose impairments do not yet meet the criteria for MCI.

A recent development in the assessment of functional capacity is that of computerized assessment strategies. Using both virtual reality and video technology, tasks have been developed that realistically simulate everyday functional tasks. Some of these tasks have demonstrated sensitivity to age related differences in healthy people, substantial correlations with cognitive test performance, and substantial separation of the performance of healthy people from various impaired populations.[47] Many real life functional tasks themselves are now performed on a computer, such as bill paying, information-seeking, and purchasing, rendering the task of the assessment creator to creating simulations that are particularly honed and relevant. Computerized tasks are also important because paper and pencil functional capacity measures, including the FCI-SF and the UPSA, have subtests that require performance of everyday tasks that may be becoming outdated, including writing paper checks and making paper check deposits. Computerized measures are more rapidly updatable with the latest technological innovations and have been shown for years to be sensitive to variation in performance in samples of healthy older community residents.[48]

For example, in the study by Atkins et al ,[48] a computerized assessment procedure called the Virtual Reality Functional Capacity Assessment (VRFCAT) was administered to 44 healthy younger adults (aged 18–30) and 41 healthy older adults ( aged 55–70). The VRFCAT is a computer-based virtual-reality measure of functional capacity that relies on a realistic simulated environment to recreate routine IADLs, with a total of 12 different demands organized into a sequence that involves a shopping task. Older adults took longer to complete the task and made more errors, as well as performed more poorly on a cognitive assessment. Importantly, composite test performance correlated r=0.66 with performance time in older adults, with correlations between each of the domains of the cognitive assessment and performance time found to be significant. Importantly for the assessment of dementia-related deficits, VRFCAT time scores were most strongly correlated with verbal episodic memory of all of the neurocognitive domains. Thus, this test shows significant potential for detection of increased ADL deficits associated with the development of new-onset MCI and has considerable evidence of convergence with cognition even in healthy populations.

Naturalistic Assessment in Real-world Environments

Another recent development in assessing early stages of cognitive change has been the naturalistic assessment of everyday functioning through the use of sensor-based technology. Advancing beyond performing realistic task simulations as described above, these strategies directly measure everyday activities in the home and the community. The advent of pervasive computing technologies deployed direct-to-home (e.g., embedded sensing and computing in the home), wireless communications, and “big data” analytics provide the route to change the current, less ecologically valid, episodic clinical trial testing paradigm. This new approach has been developed and deployed in several hundred homes of seniors followed for up to seven years by the Oregon Center for Aging & Technology (ORCATECH).[48] Thus, using an in-home array of strategically placed passive infrared motion sensors, contact sensors, physiologic vital signs (e.g. pulse, body mass index [BMI]) monitoring, medication tracking with an electronic pillbox, and telephone and computer use assessments, performance of key functions has been readily derived on a continuous or near-continuous basis. With this platform deployed to multiple homes in the community, frequent assessment of relevant outcome measures could be more sensitively assessed. These metrics include total activity in a day by location, such as particular areas of the home (e.g., bedroom, bathroom), gait speed, number of room transitions, medication adherence, social engagement (e.g., time on the telephone, time out of home), cognitively demanding functions (e.g., typical time in computer activities, recalling to take a medication), as well as more frequent (weekly) self-report of activities that cannot be simply inferred by remote sensing (e.g., rating pain, mood, falls). Importantly, aside from the absolute or mean values that are derived, minute-to-minute or day-to-day variability also becomes inherently available. This variability itself, which cannot be obtained with the current sparse measurement techniques, may be an important indicator of early change.

Using this approach, many key functions have been shown to be amenable to detecting subtle change over time in aging cohorts and those with MCI. This includes total activity,[50] gait,[51] sleep behaviors,[52] socialization,[53] computer use,[54] and medication adherence.[55] The dense (frequently captured), high dimensional (multi-domain) data lends itself to building comprehensive models of cognitive and functional change over time. Recent simulations using these data to develop more sensitive trial outcomes and better statistical approaches have been reported.[56,57] In this work, a major goal is to develop dramatically more efficient trials by creating new metrics sensitive to subtle changes in cognitive and functional outcomes using individual-specific distributions (as opposed to conventional group-norms). The ability to use this approach is enabled through the unobtrusively acquired in-home data, which allows the collection of enough data points (e.g., n=1,000 per subject) to generate individual-specific distributions of functional outcomes, such as computer usage and walking speed/variability, within a short duration of time (e.g., within 1–3 months). This then provides the ability to compare sample sizes required to achieve sufficient power to detect dementia prevention trial effects in two scenarios: 1) A conventional approach—annually assessed neuropsychological test scores modeled as a function of time using mixed effects models, and (2) a new approach using the continuous data—the likelihood of hitting subject-specific low performance thresholds modeled as a function of time using generalized mixed effects models. In the comparison of approaches, sample size estimates using the conventional approach would require approximately 2,000 subjects with a follow-up duration of three years to achieve a 30-percent effect size if the outcome is memory test scores (i.e., Logical Memory scores). If the outcome using the continuous remote sensed data is directed toward hitting an intra-individual-based low threshold of walking speed (e.g., 10th percentile of individual-specific walking speed), 263 subjects are required. For computer use (e.g., reaching the person-specific 40th percentile of low use), only 26 subjects would be required. Thus, individual-specific thresholds of low functional performance based on high-frequency, in-home monitoring data can distinguish trajectories of MCI from cognitively normal subjects with dramatically reduced sample sizes in prevention RCTs. In particular, an attractive advance might be to test candidate compounds by this method in early phases of drug development (Phases 1 and 2) to see if a clinical signal might be apparent before proceeding to large and more expensive Phase 3 trials. This approach shows great promise, especially in reducing sample sizes, which would reduce the cost of early phase studies, potentially resulting in the testing of more early-phase compounds.

The technological demands of this approach are now more minimal than ever. Individuals use their own home computers or smartphones, and their activities are passively monitored. Further, physical activity monitoring is performed through the use of wearable devices, which are in common use and continuously decreasing in price. Thus, the cost of doing studies such as these is less than the typical current strategies in patient assessment, which require visits (with payments) and study coordinators and possibly psychologists to perform the assessments.

Social Cognition and Metacognition

The assessment of social cognition and metacognition—the cognitive domains infrequently measured as part of standard cognitive evaluations—may offer another means of detecting and tracking change across the AD spectrum. Social cognition has been referred to as the means by which individuals make sense of themselves in relation to others and the world around them[58] and the processes they engage to understand or interpret the self in relation to others.[59] Metacognition, in comparison, is one’s knowledge about his or her own cognitive functioning and immediately proximal performance. While theoretically different constructs, metacognition can be considered to contribute to social cognition to the extent that preserved awareness of oneself and one’s behavior promotes social interactions. Both elements of higher-level cognition are dissociable from general cognitive abilities and contribute to important functional outcomes, such as dependence in the case of social cognition[60] and decision making capacity in the case of metacognition,[61,62] and healthcare use.[63]

Historically, both social cognition and metacognition in dementia have been evaluated based on information gathered from an informant or through clinical observation. Assessment of these areas has largely entailed evaluating elements of personality and behavior (social cognition), and symptom awareness (metacognition). In recent years, however, a number of performance-based measures have been applied, mostly in the context of research, to more precisely measure and understand the specific cognitive changes that underlie clinically observable changes in these areas. Such measures offer a promising means of identifying early signs of disease in high functioning individuals who may not yet evidence easily observable impairment, as well as more objective means of quantifying and tracking impairment. Indeed, changes in social cognition and metacognition have been reported as early as MCI,[64–67] but it is likely that subtle changes may be detectable even earlier.

Social cognition is a multi-faceted, complex construct, a characteristic that both challenges and facilitates its measurement. In one respect, the fact that many elements of cognition are considered to fall under its umbrella (e.g., perspective taking [theory of mind], emotion perception, knowledge of social norms, moral reasoning, self-monitoring, and empathy), social cognition is difficult to quickly or comprehensively measure.[68] Viewed differently, such complexity offers many opportunities to examine the integrity of social cognition. A number of studies have examined elements of social cognition in AD using performance-based measures, revealing impairments in abilities such as processing facial emotions,[69] making moral judgments,[70] and comprehending social situations.[71] Thus while elements of social cognition are generally found to be preserved in AD as compared with frontotemporal dementia,[72] they are not fully preserved nor do they appear to be fully explained by general cognitive deficits.

One of the most frequently applied performance-based measures of social cognition, the “theory of mind” (TOM) task, examines the degree to which individuals can attribute independent mental and emotional states to another—that is, to take another’s perspective.[73] Aspects of TOM performance in both AD[74,75] and MCI64–66] have been reported to be impaired in comparison with healthy older adults. Verdon et al[76] conducted a study to more closely examine the basis of TOM deficits in AD and to determine whether there was a specific deficit in reasoning about psychological intention as opposed to causality more generally. Results suggested that patients had selective impairment understanding when an individual’s deliberate actions lead to an intended goal (i.e., the factors that cause individuals to behave). Moreover, this impairment was worse in individuals with more advanced disease, and preceded impaired reasoning regarding causes of physical events. Recent work has also attempted to look at TOM in a more natural context using a referential communication task.[65] Such a task enables examination of how speakers take into account experiences and perspectives shared with others through conversation. Individuals with MCI were found to rely less on mutually shared experiences than did healthy older adults, and this was not a reflection of impaired memory for such information. Accumulating evidence, therefore, suggests that a variety of performance-based tasks assessing social cognitive processes, only a few of which were reviewed here, may offer an important means of detecting disease and tracking disease progress from its earliest stages.

The use of performance-based tasks of metacognition is another area of potential utility. Several groups have shown that individuals with AD as a whole have greater difficulty monitoring their memory performance than healthy older adults.[76–78] That is, individuals with AD are less likely to make accurate estimations regarding episodic memory performance when queried in the context of metacognitive tasks such as “feeling of knowing” (FOK) or “judgment of learning” (JOL). Such tasks generate objective metrics of memory awareness, reflecting the degree to which individuals adjust their expectations for performance in accord with actual memory performance (e.g., resolution, relative accuracy) as well as the extent to which individuals are generally under or overconfident in their estimations (e.g., calibration, absolute accuracy). Objective metacognitive tasks also enable examination of the factors that influence self-assessment and the conditions under which it is best preserved. One compelling reason to further explore the utility of metacognitive tasks for measuring and tracking disease is that impaired performance on such tasks has been shown to relate to clinically observed deficits in self-awareness,[60] a disease symptom with important implications for patient decision making, safety, and independence.

Summary and Conclusion

It is apparent that the continued development of novel methods of cognitive and performance-based assessment is needed as the focus moves toward earlier and more reliable identification and treatment of earlier stages of AD. It is possible that the most viable strategies for both early detection and sensitivity to treatment of early-stage AD will be novel tasks that have shown sensitivity to pre-MCI states and have been validated against biomarkers. The task of measuring decline in prevention trials is challenging, in that subtle declines need to be differentiated from clinical stability induced by treatment. This task will require tools with minimal retesting effects and measures that can be repeated over lengthy periods of time.

The task of functional assessment is no less challenging, but it is potentially even more important because of its direct relevance to disease and disability; only subtle deficits are likely present and the psychometric challenges in prevention trials are similar to those for cognitive deficits. Innovative observational and simulation strategies are being explored, but these will require regulatory endorsement for use in trials. As discussed by Posner et al,[1] functional co-primary measures may not be required in very early phase AD studies, but a focus on development of sensitive functional measures could also lead to their use as a single outcome measure as well.

Tests of metacognition, social cognition, and prospective memory are also newer approaches to tapping into different aspects of cognition than standard tests of episodic memory. Strategies such as controlled learning (when subjects are used as their own control) and optimizing initial encoding to enhance the potential for detection of very subtle forgetting processes appear to be promising avenues for future research.

It should be noted that the various cognitive- and performance-based tasks discussed above are not without limitation. Many of the cognitive tasks we describe target highly specific abilities, by definition missing the other components of complex cognitive constructs that may be important to assess. Moreover, challenging cognitive tasks demand some level of general cognitive abilities in order for the participant to engage in the task in a valid manner, which may make them difficult to apply at later stages of the disease process. Finally, the utility of assessing certain abilities such as social cognition and metacognition, for example, as early markers of impairment may be limited by the fact that as opposed to memory, which is quite uniformly impaired in early AD, significant heterogeneity exists in these abilities across individuals. Nonetheless, measurement of these areas would fill a current gap in assessment practice and hold promise for predicting important patient outcomes over the disease course.


This paper is a product of the “Cognitive Assessment of Early Alzheimer’s Disease in Clinical Trials” working group of the International Society for Clinical Trials and Methods (ISCTM). The executive committee of the society has endorsed this paper. The authors of the paper consist of individuals who presented at meetings, wrote and edited the manuscript, and approved its final content. Other individuals who attended these meetings contributed to the concepts and content of this paper. This paper does not reflect the opinions or endorsement of the employers of the authors or funding agencies that supported the research reviewed in this paper. Order of authorship, other than the workshop co-chairs, is presented in alphabetical order.


  1. Posner H, Curiel R, Edgar C, et al. Outcomes assessment in clinical trials of Alzheimer’s disease and its precursors: readying for short-term and long-term clinical trial needs. Innov Clin Neurosci. 2017;14(1–2):22–29
  2. Duara R, Loewenstein DA, Greig, et al. PreMCI and MCI: neuropsychological, and imaging features and progression rates. Am J Ger Psychiat. 2011;9:951–960.
  3. Espinosa A, Alegret M, Valero S, et al. A longitudinal follow-up of 550 mild cognitive impairment patients: evidence for large conversion to dementia rates and detection of major risk factors involved. J Alzheimers Dis. 2013;34:769–780.
  4. Petersen RC, Wiste HJ, Weigand SD, et al. Association of elevated amyloid levels with cognition and biomarkers in cognitively normal people from the community. JAMA Neurol. 2016;73:85–92.
  5. Katsumata Y, Mathews M, Abner EL, et al. Assessing the discriminant ability, reliability, and comparability of multiple short forms of the Boston Naming Test in an Alzheimer’s disease center cohort. Dement Geriatr Cogn Disord. 2015;39:215–227.
  6. Goldberg TE, Keefe RS, Goldman RS, et al. Circumstances under which practice does not make perfect: a review of the practice effect literature in schizophrenia and its relevance to clinical treatment studies. Neuropsychopharmacology. 2010;35:1053–1062.
  7. Goldberg TE, Harvey PD, Wesnes KA, et al. Practice effects due to serial cognitive assessment: Implications for preclinical Alzheimer’s disease randomized controlled trials. Alz Dem Diag, Assess, Dis Monitor. 2015;1:103–111.
  8. Welsh KA, Butters N, Hughes J, et al. Detection and staging of dementia in Alzheimer s disease: use of the neuropsychological measures developed for the Consortium to Establish a Registry for Alzheimer’s disease (CERAD). Arch Neurol. 1992;49:448–452.
  9. Welsh KA, Butters N, Hughes J, et al. Detection of abnormal memory decline in mild cases of Alzheimer’s disease using CERAD neuropsychological measures. Arch Neurol. 1991;48:278–281.
  10. Rolstad S, Nordlund A, Eckerström C, et al. High education may offer protection against tauopathy in patients with mild cognitive impairment. J Alzheimers Dis. 2010;21:221–228.
  11. O’Bryant SE, Lacritz LH, Hall, et al. Validation of the new interpretive guidelines for the clinical dementia rating scale sum of boxes score in the national Alzheimer’s coordinating center database. Arch Neurol. 2010;67:746–749.
  12. Harvey PD, Heaton RK, Palmer BA, et al. Stability of cognitive impairment in elderly schizophrenic patients receiving conventional antipsychotic treatment. Am J Psychiat. 2005;162:110–117.
  13. Loewenstein DA, Greig-Custo MT, Schinka JA, et al. An Investigation of PreMCI: Subtypes and Longitudinal Outcomes. Alzheim Dement. 2012;8:172–179.
  14. Loewenstien DA, Acevedo A, Agron J, Duara R. Vulnerability to proactive semantic interference and progression to dementia in older adults with mild cognitive impairment (MCI). Dement Geriatr Cogn Disord. 2007;24:363–368.
  15. Loewenstein DA, Acevedo A, Luis C, et al. Semantic interference deficits and the detection of mild Alzheimer’s disease and mild cognitive impairment without dementia. J Int Neuropsychol Soc. 2004;10:91–100.
  16.  Snitz B, Loewensetin DA, Saxton K, Ganguli M. A novel approach to assessing memory at the population level: vulnerability to semantic interference. Int Psychogeriatrics. 2010;1:1–10.
  17. Buschke H, Silwinski M, Kuslansky G, Lipton RB. Aging, encoding specificity, and memory change in the Double Memory Test. J Int Neuropsychol Soc. 1995;5:483–493.
  18. Buschke H, Silwinski MJ, Kuslanski G, Lipton RB. Diagnosis of early dementia by the double memory test. Neurology. 1997;48:989–996.
  19. Salmon DP, Bondi MW. Neuropsychological assessment of dementia. Ann Rev Psychol. 2009;60:257-
  20. Thompson C, Henry JD, Rendell PG, et al. Prospective memory function in mild cognitive impairment and early dementia. J Int Neuropsychol Soc. 2010;16:318–325.
  21. Sp?ndola L, Brucki SMD. Prospective memory in Alzheimer’s disease and mild cognitive impairment. Dement Neuropsychol. 2011;5:64–68.
  22. Hernandez-Cardenache R, Burguera L, Acevedo A, et al. Evaluating different aspects of prospective memory in amnestic and nonamnestic mild cognitive impairment. ISRN Neurol. 2014:805929.
  23. Cavedo E, Lista S, Khachaturian Z, et al. The road ahead to cure Alzheimer’s disease: development of biological markers and neuroimaging methods for prevention trials across all stages and target populations. J Prev Alzheimers Dis. 2014;1(3):181–202.
  24. Albert MS, Dekosky ST, Dickson D, et al. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging and Alzheimer’s Association workgroup. Alzheimers Dement. 2011;7:270–279.
  25. Curiel R, Crocco E, Acevedo A, et al. A new scale for the evaluation of proactive and retroactive interference in mild cognitive impairment and early Alzheimer’s disease. J Aging Sci. 2014.1:
  26. Crocco E, Curiel R, Acevedo A, et al. An evaluation of deficits in semantic cueing and proactive and retroactive interference as early features of Alzheimer’s disease. Am J Geriatric Psychiat. 2014;22:889–897.
  27. Loewenstein DA, Curiel RE, Greig-Custo MT, et al. The Relationship between a novel test of semantic interference (LASSI-L) and global and regional accumulation of amyloid in the brains of community-dwelling elders. Presented at Alzheimer’s Association Annual Conference. Washington, DC: 18–23 July 2015.
  28. Parra MA, Abrahams S, Logie RH, et al. Visual short-term memory binding deficits in familial Alzheimer’s disease. Brain. 2010;133:2702–2713.
  29. Parra MA, Sala SD, Abrahams S, et al. Specific deficit of colour-colour short-term memory binding in sporadic and familial Alzheimer’s disease. Neuropsychologia. 2011;49:1943–1952.
  30. Della Sala S, Parra MA, Fabi K, et al. Short-term memory binding is impaired in AD but not in non-AD dementias. Neuropsychologia. 2012;50:833–840.
  31. Rentz DM, Parra Rodriguez MA, Amariglio R, et al. Promising developments in neuropsychological approaches for the detection of preclinical Alzheimer’s disease: a selective review. Alzheimers Res Ther. 2013;5:58–68.
  32. Marshall GA, Amariglio RE, Sperling RA, Rentz DM. Activities of daily living: Where do they fit in the diagnosis of Alzheimer’s disease? Neurodegener Dis Manag. 2012;2:483–491.
  33. Farias ST, Chou E, Harvey DJ, et al. Longitudinal trajectories of everyday function by diagnostic status. Psychol Aging. 2013;28:1070–1075.
  34. Marson D. Investigating functional impairment in preclinical Alzheimer’s disease: potential measure characteristics and methodology. J Prevention Alzheimer’s Disease. 2015;2:4–6.
  35. Griffith HR, Belue K, Sicola A, et al. Impaired financial abilities in mild cognitive impairment: a direct assessment approach. Neurology. 2003;60:449–457.
  36. Triebel K, Martin R, Griffith HR, et al. Declining financial capacity in mild cognitive impairment: a one-year longitudinal study. Neurology. 2009;73:928–934.
  37. Jefferson AL, Byerly LK, Vanderhill S, et al. Characterization of activities of daily living in individuals with mild cognitive impairment. Am J Geriatr Psychiat. 2008;16:375–383.
  38. Tomaszewski Farias S, Mungas D, et al. MCI is associated with deficits in everyday functioning. Alzheim Dis Assoc Disord. 2006;20:217–223.
  39. Marson DC, Sawrie SM, Snyder S, et al. Assessing financial capacity in patients with Alzheimer disease: a conceptual model and prototype instrument. Arch Neurol. 2000;57:877–884.
  40. Marson DC, Triebel KL, Gerstenecker A, et al. Detecting declining financial skills in preclinical Alzheimer’s disease: the Financial Capacity Instrument–Short Form. Presented at the 10th Annual Autumn Conference of the International Society for CNS Clinical Trials and Methodology (ISCTM). Boston, MA: 6–8 October 2014.
  41. Marson D, Triebel K, Gerstenecker A, et al. Detecting declining functional skills in preclinical Alzheimer’s disease: the Financial Capacity Instrument—short form. J Prevent Alzheim Dis. 2014;1:265.
  42. Widera E, Steenpass V, Marson D, Sudore R. Finances in the older patient with cognitive impairment: “He didn’t want me to take over.” JAMA. 2011;305(7):698–706.
  43. Patterson TL, Goldman S, McKibbin CL, et al. UCSD performance-based skills assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophr Bull. 2001;27:235–245.
  44. Goldberg TE, Koppel J, Keehlisen L, et al. Performance-based measures of everyday function in mild cognitive impairment. Am J Psychiat. 2010;167:845–845.
  45. Gomar JJ, Harvey PD, Bobes-Bascaran MT, et al. Development and cross-validation of the UPSA short form for the performance-based functional assessment of patients with mild cognitive impairment and Alzheimer disease. Am J Geriatr Psychiatry. 2011;19:915–922.
  46. Harvey PD, Loewenstein DA, Czaja SJ. Hospitalization and psychosis: influences on the course of cognition and everyday functioning in people with schizophrenia. Neurobiol Dis. 2013;53:18–25.
  47. Ruse SA, Harvey PD, Davis VG, et al. Virtual reality functional capacity assessment in schizophrenia: preliminary data regarding feasibility and correlations with cognitive and functional capacity performance. Schizophr Res Cogn. 2014;1:21–26.
  48. Atkins AS, Stroescu I, Spagnola NB, et al. Assessment of age-related differences in functional capacity using the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). J Prev Alzheimers Dis. 2015;2:121–127.
  49. Kaye JA, Maxwell SA, Mattek N, et al. Intelligent systems for assessing aging changes: home-based, unobtrusive, and continuous assessment of aging. J Gerontol B Psychol Sci Soc. Sc 2011i. i180–i190.
  50. Hayes TL, Abendroth F, Adami A, et al. Unobtrusive assessment of activity patterns associated with mild cognitive impairment. Alzheim Dement. 2008;4:395–405.
  51. Dodge HH, Mattek NC, Austin D, et al. In-home walking speeds and variability trajectories associated with mild cognitive impairment. Neurology. 2012;78:1946–1952.
  52. Hayes TL, Riley T, Mattek N, et al. Sleep habits in mild cognitive impairment. Alzheimer Dis Assoc Disord. 2014;28:145–150.
  53. Petersen J, Austin D, Kaye JA, et al. Unobtrusive in-home detection of time spent out-of-home with applications to loneliness and physical activity. IEEE J Biomed Health Inform. 2014;18:1590–1596.
  54. Kaye J, Mattek N, Dodge HH, et al. Unobtrusive measurement of daily computer use to detect mild cognitive impairment. Alzheim Dement. 2014;10:10–17.
  55. Hayes TL, Larimer N, Adami A, Kaye JA. Medication adherence in healthy elders: small cognitive changes make a big difference. J Aging Health. 2009;.
  56. Dodge HH, Kaye J, Zhu J, et al. Use of intra-individual distributions of daily acquired home-based measures increases RCT sensitivity. Alzheim Dement. 2014;10:P204.
  57. Dodge HH, Zhu J, Mattek NC, et al. Use of high-frequency in-home monitoring data may reduce Sample sizes needed in clinical trials. PLoS ONE. 2015;10(9):e0138095. doi:10.1371/journal.pone.0138095
  58. Fiske ST. Social cognition and social perception. Ann Rev Psychol. 1993;44:155–194.
  59. Forbes CE, Grafman J. The role of the human prefrontal cortex in social cognition and moral judgment. Annu Rev Neurosc. 2010;33:299–324.
  60. Cosentino S, Zahodne L, Brandt J. et al. Social cognition in Alzheimer’s disease: a separate construct contributing to dependence. Alzheim Dement. 2014;10:818–826.
  61. Cosentino S, Metcalfe J, Cary M, et al. Memory awareness influences everyday decision making capacity in Alzheimer’s disease. Int J Alzheim Dis. 2011; Article ID 483897.
  62. Karlawish JH, Casarett DJ, James BD, et al. The ability of persons with Alzheimer’s disease (AD) to make a decision about taking an AD treatment. Neurology. 2005;64:1514–1519.
  63. DeFeis B, Zhu C, Azar M, et al. The cost of disordered awareness: An examination of medicare use in dementia. Alzheim Dement. 2015;11:P597.
  64. Baglio F, Castelli I, Alberoni M, et al. Theory of mind in amnestic mild cognitive impairment: an FMRI study. J Alzheim Dis. 2012;29:25–37.
  65. Moreau N, Rauzy S, Bonnefoi B, et al. Different patterns of theory of mind impairment in mild cognitive impairment. J Alzheim Dis. 2015;45:581–59.
  66. Poletti M, Bonuccelli U. Alteration of affective theory of mind in amnestic mild cognitive impairment. J Neuropsychol. 2013;7:121–131.
  67. Pinkham AE, Penn DL, Green MF, et al. The Social Cognition Psychometric Evaluation (SCOPE) study: results of the expert survey and RAND panel. Schizophr Bull. 2014;40:813–823.
  68. Miller LA, Hsieh S, Lah S, et al. One size does not fit all: face emotion processing impairments in semantic dementia, behavioural-variant frontotemporal dementia and Alzheimer’s disease are mediated by distinct cognitive deficits. Behav Neuro. 2012;25:53–60.
  69. Torralva T, Dorrego F, Sabe L, et al. Impairments of social cognition and decision making in Alzheimer’s disease. Int Psychogeriatr. 2000;12:359–368.
  70. Cavallo M, Enrici I, Adenzato M. The comprehension of social situations in a small group of patients with frontotemporal dementia and Alzheimer’s disease. Acta Neuropsychologica. 2011;9:167–176.
  71. Shany-Ur, T Poorzand P, Grossman SN, et al. Comprehension of insincere communication in neurodegenerative disease: lies, sarcasm, and theory of mind. Cortex. 2012;48:1329–1341.
  72. Baron-Cohen S, Ring HA, Wheelwright S, et al. Social intelligence in the normal and autistic brain: an fMRI study. Eur J Neurosci. 1999;11:1891–1898.
  73. Freedman M, Binns MA, Black SE, et al. Theory of mind and recognition of facial emotion in dementia: challenge to current concepts. Alzheimer Dis Assoc Disord. 2013 ;27(1):56-61.
  74. Gregory C, Lough S, Stone V, et al. Theory of mind in patients with frontal variant frontotemporal dementia and Alzheimer’s disease: theoretical and practical implications. Brain. 2002;125:752–764.
  75. Verdon CM, Fossati P, Verni M, et al. Social cognition: an early impairment in dementia of the Alzheimer type. Alzheimer Dis Assoc Disord. 2007;21:25–30.
  76. Cosentino S, Metcalfe J, Butterfield B, Stern Y. Objective metamemory testing captures awareness of deficit in Alzheimer’s disease. Cortex. 2007;43:1004–1019.
  77. Perrotin A, Belleville S, Isingrini M. Metamemory monitoring in mild cognitive impairment: evidence of a less accurate episodic feeling-of-knowing. Neuropsychologia. 2007;45:2811–2826.
  78. Souchay C. Metamemory in Alzheimer’s disease. Cortex. 2007;43:987–1003.

Tags: , , , , ,

Category: Alzheimer's Disease, Cognition, Current Issue, Dementia, Drug Development, Evaluations, Neurology, Patient Assessment, Review, Scales, Trial Methodology

Leave a Reply