Beyond Reliability: Biomarkers and Validity in Psychiatry

| January 8, 2008 | 0 Comments

by Ronald Pies, MD

Dr. Pies is Professor of Psychiatry, SUNY Upstate Medical University, Syracuse, New York; and Clinical Professor of Psychiatry, Tufts USM, Boston, Massachusetts.


The validity of psychiatric diagnosis rests in part on a demonstration that identifiable biomarkers exist for major psychiatric illnesses. Recent evidence supports the existence of several biomarkers or endophenotypes for both schizophrenia and bipolar disorder. As we learn more about how these biomarkers relate to the symptoms, course, and treatment response of major psychiatric disorders, the “objectivity” of psychiatric diagnosis will increase. However, psychiatry is and will remain a clinically based discipline, aimed at comprehensively understanding and relieving human suffering.

Key Words

psychiatric diagnosis, validity, objectivityintroduction


In a previous commentary,[1] I made the case that the interrater reliability (“kappa”) of psychiatric diagnosis is often comparable to that in other medical specialties; and that, insofar as this is true, psychiatric diagnosis partakes of “objectivity.” But even readers who might accept this conclusion would point out that reliability and validity are two different issues. As McHugh and Slavney succinctly put it, “Reliability is the verification of observations. Validity is the verification of presumptions.”[2] After all, two near-sighted observers may “agree” that a large, four-legged mammal is a unicorn, but that hardly proves the existence of unicorns. Indeed, there is a well-worn argument promulgated by certain critics of psychiatry purporting to show that psychiatric disorders are not “real diseases” at all; rather, these critics argue, they are more like unicorns. I believe this argument is fallacious on logical, linguistic, and historical grounds, as I and others have tried to show in several extensive reviews.[3–5] Nevertheless, the validity of several psychiatric diagnoses remains a matter of considerable controversy, even among psychiatrists who generally support the conceptual framework of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV). To cite but one example, the validity of the diagnosis of oppositional defiant disorder has been both disputed[6] (correctly so, in my view) and defended.[7]

Strictly speaking, reliability ought to be imputed to diagnostic procedures, not to diagnoses per se.[8] Validity, on the other hand, signifies something tied fundamentally to the diagnosis itself. Validity has to do with the ontological “reality” of a putative condition or disorder, whether, unlike the unicorn, our diagnoses “…correspond to any real entities…[or] diseases”[9] beyond our own imagining. Although this can easily become a metaphysical rather than a medical question, physicians and scientists have approached the construct of validity in several concrete ways. In this commentary, I focus on the concept of biomarkers and endophenotypes, homing in on two major psychiatric illnesses: bipolar disorder and schizophrenia.

Biomarkers and Endophenotypes

S. King, a pathologist by training, once observed that, “A [disease] pattern has reasonable stability only when its criteria are sharp; its elements cohere; and its utility in clarifying experience remains high.”[10] The third element in King’s triad is of particular importance to establishing validity in clinical medicine. Our “experience” of a putative disease includes, for example, our observations of its typical course and prognosis, its response to specific treatments, its familial pattern, and its association with specific, measurable, physiochemical and/or neurobehavioral findings.

This last component is often placed under the rubric of biological markers or biomarkers. A particular subtype of biomarker is known as an endophenotype.[10,11] As developed by Gottesman and others, an endophenotype (EP) may be defined as a discrete, heritable, genetically determined intervening variable that is often part of an illness, but which is not readily discerned by routine clinical examination of the patient.[12]

Ideally, EPs should be “state-independent;” that is, they should be detectable whether or not the patient is acutely ill, though sometimes a “challenge test” may be necessary to bring out the endophenotypic trait.[11] So, for example, if patients with schizophrenia consistently demonstrated a particular type of heritable abnormality in motor function, detectable only upon careful neurological examination, that might qualify as an endophenotype for the disease. Putative EPs may include neurocognitive abnormalities, regional brain abnormalities, or abnormal metabolic/ physiologic functions, among other findings.

There is a widespread misperception, in my view, that psychiatry is not even close to having biomarkers for most major psychiatric disorders—a view I often hear even from psychiatrists. In fact, recent research suggests that we are fast approaching the validation of specific biomarkers for several disorders, especially schizophrenia and bipolar disorder.

Biomarkers in Schizophrenia

Although the neuropathology of schizophrenia is not fully elucidated, research over the past two decades has clarified a great deal. For example, in patients with schizophrenia (compared with controls), there is now growing evidence of (1) significantly-reduced anterior cingulate volumes;[13] (2) enlarged lateral and third ventricular volumes;[14] and (3) white matter abnormalities that may lead to disrupted “connectivity” between different brain regions.[15] Though there is no “smoking gun”—no single abnormality that appears consistently in the brains of all schizophrenic patients—the aforementioned meta-analyses[13–15] point to several abnormalities frequently identified in schizophrenia. Indeed, it would be passing strange to discover that a complex neuropsychiatric syndrome like schizophrenia, with its various subtypes, is correlated with only a single brain abnormality. We certainly would not expect this of other complex brain disorders, such as Alzheimer’s disease or Lewy-Body dementia.

Several other putative biomarkers or endophenotypes have also been proposed for schizophrenia. A complete review of these is clearly beyond the scope of this commentary. However, one especially promising marker merits brief discussion. Neuregulin-1 (NRG1) was recently identified as a susceptibility gene for schizophrenia, in an Icelandic population.[16] Specifically, two genetic markers on both sides of the neuregulin-1 gene, on the short arm of chromosome 8, revealed a highly significant association with schizophrenia. Now, many “biomarker” genes have come and gone in the checkered history of schizophrenia research. But in the case of NRG1, the finding in an Icelandic population was subsequently replicated in a second (Scottish) population.[17] Even more striking is the tight functional connection between NRG1 and various neurotransmitter pathways that have long been scrutinized in schizophrenia, e.g., pathways affecting glutamate and GABA.[17] While more research is needed, neuregulin-1 is emerging as a promising biomarker in schizophrenia.

Besides the NRG1 biomarker, perhaps the most convincing and consistent data in schizophrenia relate to a phenomenon called abnormal smooth-pursuit eye movements (SPEMs). Essentially, the human eye can track a moving target (such as the clinician’s finger) using one of two neuronal systems: one that closely matches the foveal area of the retina to the visual image of the target (“smooth” system) and one that uses “jumps” or saccadic movements to bring the target image to the fovea. Several independent groups have found that in subjects with schizophrenia there is a failure to inhibit the second, saccadic system, resulting in the intrusion of saccades during the smooth pursuit task.[18,19] Moreover, compared to the healthy subjects, patients with schizophrenia exhibit greater activity in both posterior hippocampi and the right fusiform gyrus during smooth pursuit eye movements. This observation is consistent with the hypothesis that inhibitory interneurons in the hippocampus are dysfunctional in schizophrenia.[20]

Tregallas, et al., conclude that “…smooth pursuit eye movement abnormalities are among the most reproducible physiologic abnormalities associated with schizophrenia.”[20]
Furthermore, the abnormality is not an artifact of medication and has also been observed in at-risk individuals, such as the non-schizophrenic relatives of patients with schizophrenia.[21] It is disappointing that these striking SPEM findings have received so little attention from the lay media or, indeed, from most psychiatrists. Perhaps this will change if we can develop an accurate, office-based method of detecting abnormal SPEMs; at present, specialized optical equipment is required.[20,21]

Biomarkers in Bipolar Disorder

Although biomarkers are not as well established in bipolar disorder as in schizophrenia, we are beginning to find a set of overlapping brain abnormalities that characterize many patients with bipolar disorder. One recent review of proton magnetic resonance spectroscopy findings examined 22 studies involving 328 adult bipolar and 349 control subjects.[22] Taken in toto, the studies pointed to focal abnormalities of N-acetylaspartate (NAA), choline, and glutamate/glutamine in bipolar disorder. In regional terms, the studies found abnormalities in dorsolateral prefrontal cortex (DLPFC), cingulate cortices, hippocampus, and basal ganglia.

Consistent with and complementing these findings, structural and functional neuroimaging studies point to the disruption of brain networks in patients with bipolar disorder. To oversimplify greatly, bipolar disorder may involve a “failure to communicate” among prefrontal, subcortical, and limbic regions of the brain. There may also be a failure of prefrontal regions to dampen excessive activity in anterior limbic system structures, such as the amygdala and thalamus—a kind of “top to bottom” disinhibition problem.[23] In an elegant chapter on endophenotypes in bipolar disorder, Doran[11] has summarized much of the known literature, and some of these more salient biomarkers discussed are shown in Table 1.

Finally, recent research suggests that probably there is not a “wall of separation” between schizophrenia and bipolar disorder, from the standpoint of neurocircuitry problems. Indeed, in a recent review, Keshavan, et al., opined “…massive disruptions in circuitry may lead to extreme deficits in cognition and thought disorder that are observed in…schizophrenia…on the other hand, more selective disruptions in frontal, subgenual and limbic circuitry may underlie disorders of mood regulation, such as…bipolar disorder and depression.”[24]


It is reasonable to hypothesize that the more uniform a psychiatric condition’s course and prognosis; the more predictably it responds to specific treatments; and the more it can be linked to well defined biomarkers, the greater the validity of the diagnosis.[2,9] Moreover, the more our diagnostic procedures take all these factors into account, the greater our level of diagnostic “objectivity.”

I do not want to suggest, however, that the use of biomarkers is either necessary or sufficient for the clinical diagnosis of psychiatric disorders. At most, these are investigative procedures that hold promise as confirmatory tests. Careful, clinical observation and history-taking, together, are still the gold standard of psychiatric diagnosis. This will remain so, in my view, even as we discover accurate biomarkers for psychiatric disorders. This position is commensurate with that of our colleagues in neurology, who insist, to this day, that epilepsy remains a clinical diagnosis, even though magnetic resonance imaging (MRI) and electroencephalogram (EEG) findings are often, but not always, confirmatory. As neurologist Dr. Anthony Marson succinctly puts it, “Epilepsy is a clinical diagnosis….the diagnosis is not based upon the results of tests, but upon an accurate description of the attacks.”[25] Similarly, an accurate formulation of the patient’s history, signs, and symptoms will always remain at the core of psychiatric diagnosis.

The mere existence of biomarkers in psychiatric illness does not mean we should ignore the cultural, psychosocial, and existential components of our patients’ problems,9 or attribute their psychopathology to biochemical factors alone. Nonetheless, accurate biomarkers, along with more reliable and valid disease criteria, will help psychiatry achieve greater objectivity in diagnosis. Even more promising, biomarkers may soon help us diagnose psychiatric disorders in their earliest stages, potentially enhancing the care of our patients.

1. Pies R. How “objective” are psychiatric diagnoses? (Guess again). Psychiatry (Edgemont) 2007;4(10):18–22.
2. McHugh PR, Slavney PR. The Perspectives of Psychiatry. Baltimore, MD: Johns Hopkins University Press, 1986.
3. Pies R. On myths and counter-myths: More on Szaszian fallacies. Arch Gen Psychiatry 1979;33:139–44.
4. Pies R. Moving beyond the “myth” of mental illness. In: Schaler JA (ed). Szasz Under Fire. Chicago, IL: Open Court, 2004:27–53.
5. Roth M. Schizophrenia and the theories of Thomas Szasz. Br J Psychiatry 1976;129:317–26.
6. Mota-Castillo M. The crisis of overdiagnosed ADHD in children. Psychiatr Times 2007;24(8):12–13.
7. Keenan K, Wakschlag LS, Danis B, et al. Further evidence of the reliability and validity of DSM-IV ODD and CD in preschool children. J Am Acad Child Adolesc Psychiatry 2007 Apr;46(4):457-68.
8. First M. Personal communication. October 31, 2007.
9. Ghaemi SN. The Concepts of Psychiatry. Baltimore, MD: Johns Hopkins University Press, 2003.
10. King LS. What is disease? Philos Sci 1954;21:193–203.
11. Doran CM. The Hypomania Handbook. Philadelphia, PA: Wolters Kluwer/Lippincott Williams & Wilkins, 2008.
12. Braff DL, Freedman R, Schork NJ et al. Deconstructing schizophrenia: An overview of the use of endophenotypes in order to understand a complex disorder. Schizophr Bull 2007;33:21–32.
13. Baiano M, David A, Versace A, et al. Anterior cingulate volumes in schizophrenia: A systematic review and a meta-analysis of MRI studies. Schizophr Res 2007;93:1–12. Epub 2007 Mar 30.
14. Vita A, De Peri L, Silenzi C, et al. Brain morphology in first-episode schizophrenia: A meta-analysis of quantitative magnetic resonance imaging studies. Schizophr Res 2006;15:75-88. Epub 2005 Dec 27.
15. Kubicki M, McCarley R, Westin CF et al. A review of diffusion tensor imaging studies in schizophrenia. J Psychiatr Res 2007;41:15–30. Epub 2005 Jul 14.
16. Stefansson H, Sigurdsson E, Steinthorsdottir V, et al. Neuregulin 1 and susceptibility to schizophrenia. Am J Hum Genet 2002;71:877–92.
17. Stefansson H, Sarginson J, Kong A et al. Association of neuregulin 1 with schizophrenia confirmed in a Scottish population. Am J Hum Genet Am J Hum Genet. 2003;72(1):83–7. Epub 2002 Dec 11.
18. Levy DL, Holzman PS, Matthysse S, et al. Eye tracking dysfunction and schizophrenia: A critical perspective. Schizophr Bull 1993;19:461–536.
19. Hommer DW, Clem T, Litman R, Pickar D. Maladaptive anticipatory saccades in schizophrenia. Biol Psychiatry 1991;30:779–94.
20. Tregellas JR, Tanabe JL, Miller DE et al. Neurobiology of smooth pursuit eye movement deficits in schizophrenia: An fMRI study. Am J Psychiatry 2004;161:315–21.
21. Hong LE, Mitchell BD, Avila MT, et al. Familial aggregation of eye-tracking endophenotypes in families of schizophrenic patients. Arch Gen Pasychiatry 2006;63:259–64.
22. Yildiz-Yesiloglu A, Ankerst DP. Neurochemical alterations of the brain in bipolar disorder and their implications for pathophysiology: A systematic review of the in-vivo proton magnetic resonance spectroscopy findings. Prog Neuropsychopharmacol Biol Psychiatry 2006;30:969–95.
23. Strakowski SM, Delbello MP, Adler CM. The functional neuroanatomy of bipolar disorder: A review of neuroimaging findings. Mol Psychiatry 2005;10:105–16.
24. Keshavan MS, Diwadkar V, Rosenberg DR. Developmental biomarkers in schizophrenia and other psychiatric disorders: common origins, different trajectories? Epidemiol Psichiatr Soc 2005;14:188–93.
25. Marson A. What is epilepsy? Based on a text by Dr Per Rochat and Dr Jørgen Alving. Available at: Access date: December 18, 2007.

Category: Commentary, Psychiatry, Reliability of Diagnoses

Leave a Reply