by Philip D. Harvey, PhD
Dr. Harvey is Professor of Psychiatry and Behavioral Sciences, Emory University School of Medicine, Atlanta, Georgia
Print citation
Psychiatry (Edgemont) 2007;4(12)29-31
As we discussed in this column before,[1] the NIMH-funded MATRICS initiative led to the establishment of a MATRICS consensus cognitive assessment battery (MCCB) and a regulatory pathway that was approved by consensus of the NIMH, the Food and Drug Association (FDA), and a large academic consortium.[2] The features of the regulatory pathway are of interest because there is a specification of both the duration of the registration trial (6 months) and of the need for an additional “co-primary” outcomes measure. Based on the FDA experience with registration trials for cognitive and functional disorders in Alzheimer’s disease and Parkinson’s disease, this co-primary measure is designed to be an additional assessment that would determine whether the identified cognitive changes had clinical relevance.
While this type of outcome is not difficult to collect in registration trials in Alzheimer’s disease, where there is often an informant who is very familiar with the patient, there are some challenges to collection of such an outcomes measure in schizophrenia. Noting that there is really no consensus in the field as to how best to gauge the clinical relevance of a cognitive improvement, the NIMH, in conjunction with a consortium of industry partners, has launched two parallel initiatives, referred to as MATRICS-C-T. The T part is the systematic translation of the MCCB into other languages and the C portion, the development of a co-primary outcomes measure for clinical trials, will be the focus of the column.
There are several major considerations that must be kept in mind during the development of a co-primary measure that is aimed at use in clinical treatment studies of cognitive impairment. These concerns include both the content of the assessment and the method of the assessment. Content issues are important, not only in terms of the general domains of functional outcomes (i.e., social, independent living, self-care, and vocational), but in terms of the specific details and focus. In a six-month clinical trial, major milestones, such as establishment of a permanent residence, marriage, or full-time employment, are not realistic outcome measures of interest and they may not be realistic to expect at all. Regardless of the improvement in cognitive functioning realized through clinical interventions, other factors are related to attainment of these everyday outcomes. These include opportunities (availability of work), disincentives (disability payments and health insurance contingencies), and other personal and familial characteristics (such as motivation and awareness of illness and associated limitations).
All of these additional factors relate to the likelihood that individuals will actually perform up to their potential in everyday functioning. However, treatment interventions aimed at cognitive impairments are much more likely to impact the ability to perform skilled acts, not the likelihood that they will be performed. In other words, these interventions will improve competence, not necessarily performance. As a result, co-primary measures must also address improvements in competence and not examine aspects of everyday outcomes that are influenced by factors other than the ability to perform; otherwise, the measures may underestimate the magnitude of treatment-related changes in a confounded assessment.
In that context, the MATRICS-C workgroup is now examining several different potential co-primary measures. These potential measures fall under two general domains: Reports of performance generated by the patient and various informants and performance-based skills competence measures. The process of identification of the best measure is proceeding in the same way that the MCCB was developed. First there has been an extensive nomination process, with a scientific committee working to collect all of the available information for each nominated scale. Second will be a RAND panel, where a group of expert panelists will evaluate the nominated scales on several dimensions, including user-friendliness, portability, and brevity. Third will be a multisite study, which will examine the correlation between each of the scales approved by the RAND panel and the MCCB, as well as various clinical symptom indices.
One of the major challenges in the development of a co-primary measure for the assessment of cognitive and functional disability is the consistent finding that people with schizophrenia generate self-evaluations of their cognitive and functional deficits that are inconsistent with both the reports of other informants who know them well and with data based on their performance on objective measures. For example, Keefe, et al.,[3] showed that reports from an informant regarding cognitive impairment on the part of people with schizophrenia were much more strongly correlated with the patient’s objective test performance than the patient’s self report of their own deficits. In fact, the patient’s report of his or her cognitive deficits was actually correlated close to 0 with his or her performance. Similarly, Bowie, et al.,[4] reported that patients’ self-reports of their real-world functional disability were very poorly correlated with ratings on the same domains with the same rating scale generated by their case managers. In both the Keefe, et al., and Bowie, et al., studies, the best single correlate of performance on neuropsychological tests were scores on a performance-based measure of everyday living skills. Further, the reports of an individual who is quite familiar with the person with schizophrenia was correlated with measures of NP performance and performance on measures of competence in everyday living skills. Thus, these data suggest that informant reports can be reliably associated with NP performance and that performance-based measures of everyday living skills are also strong candidates as correlates of NP performance.
There will be two problems that the MATRICS-C initiative will not be able to solve, and they reflect intrinsic limitations of informant report and performance-based measures of functional capacity. The informant report measure that is the best of the group will still not work if the informant is not familiar with the patient. As suggested by the results of several studies described above, a simple transcription of the patient’s self-report is likely to be poorly correlated with NP performance. Any performance-based measure will still not be measuring the likelihood that an individual with schizophrenia performs the skills of interest. Thus, improvements in the ability to perform well on tests of financial competence will not necessarily translate directly into being able to assume more financial responsibility. As noted above, it is not assumed that cognitive enhancement and associated changes in functional capacity are guaranteed to improve real-world outcomes.
There are likely to be several positive effects of MATRCS-C. One will be the selection of a co-primary outcomes measure that is related both to everyday outcomes and to NP performance. This will improve treatment studies by increasing the chances that improvements on NP tests will correlate with changes on the co-primary measure selected. The second clear benefit will be that the assessment measure selected will be clinically useful to assess everyday functioning potential and NP performance and will be brief and useful in a variety of contexts. As a result, the selected outcomes measure will also be quite likely to have substantial clinical usefulness and be useful to a wide range of mental health professionals.
In summary, the MATRICS-C initiative will produce a validated measure of the everyday living skills that will be brief, correlated with NP performance and everyday outcomes, and designed for sensitivity to change. This outcome will likely make studies that use the MCCB more likely to detect important improvements and treatment may itself be broadly clinically useful. Such an outcome would increase the likelihood that functional assessments would be performed regularly and reliably by psychiatric clinicians and will also increase awareness of the importance of assessment of functional disability in schizophrenia.
References
1. Harvey PD. Current status of the MATRICS/ TURNS initiative. Psychiatry 2006;3(11):24–33.
2. Buchanan RW, Davis M, Goff D, et al. A summary of the FDA-NIMH-MATRICS workshop on clinical trial design for neurocognitive drugs for schizophrenia. Schizophr Bull 2005;31:5–19.
2. Keefe RSE, Poe M, Walker TM, et al. The schizophrenia cognition rating scale: An interview-based assessment and its relationship to cognition, real-world functioning, and functional capacity. Am J Psychiatry 2006;163:426–32.
3. Bowie CR, Twamley EW, Anderson H, et al. Self-assessment of functional status in schizophrenia. J Psychiatr Res 2007;41:1012–18.
