Regarding objectivity of psychiatric diagnoses

Dear Editor:
In response to Dr. Bennett Cohen’s observation [Regarding objectivity of psychiatric diagnoses (Letters). Psychiatry (Edgemont) 2007;4(12):22–3] that structured clinical interviews came into use because they already had a high rate of interrater reliability, Dr. Ronald Pies says he does not think this is so.

Dr. Pie’s reasoning appears to be that since the actual “kappa” is derived from interviewers doing structured interviews without knowing in advance how they will rate, if there is high concordance between the interviewers, then high interrater reliability is an independent factor that is unrelated to the way the questions in the structured interview were structured.

In reality, the structured interviews subtly—actually quite blatantly—screen out all forms of psychiatric enquiry where the question cannot be asked in an identical and limited fashion, and where the answers obtained would be too complex or elaborate to be placed into a few preordained categories. This is easier to demonstrate from actual communications of patients than by splitting theoretical hairs.

The very day I read Drs. Cohen’s and Pies’s letters, a patient made the following complaint: She was hearing her recently expired father-in-law’s voice calling her a “horse’s ass.” All structured interviewers would have reliably rated this symptom as an auditory hallucination and, with the help of a few more standard questions, would have had no difficulty in diagnosing it as a symptom of her paranoid schizophrenia. To illustrate how remarkably different psychiatric diagnoses are from the physical ones, it would not be inappropriate to add that based upon my knowledge of this patient for the last 13 years, I can say that she has also met at various times criteria for the following additional diagnoses: catatonic, residual, and undifferentiated schizophrenia; schizoaffective disorder; bipolar disorder with psychosis; obsessive compulsive disorder; conversion reaction; depersonalization disorder; panic disorder; phobias; alcoholism; eating disorders; morbid obesity; and multiple sleep disorders.

But the clinical essence of her hallucination had very little to do with any of these diagnoses, and none of the existing structured interviews would have even attempted to elicit the essence of this hallucination, let alone bothered with the issue of interrater reliability. It came to light only after a series of further questions and other communications that were directed to her in an unstructured fashion, and her initial answers were not accepted on face value.

On asking where exactly the phrase “horse’s ass” was taken, the patient first flatly claimed that she was unfamiliar with the expression. On pointing out that nothing can come to consciousness from inside the mind unless it was an external perception once, she recalled that her husband had, on occasion, told her that his father would use that phrase toward his children when they were young. She, however, denied that her father-in-law had ever used that phrase against her. After a little more non-specific enquiry, she suddenly added that she also had visual hallucinations of seeing him with the snout of a donkey alternating with that of a pig. When asked for associations to this hallucination, she admitted that she had never liked her father-in-law because he would make fun of her at family gatherings, which everybody told her to ignore as that was his way of showing affection. But after his death, she had been wondering whether such ridiculing was less a form of affection and more the behavior of an ass and a pig. It is obvious that his death had given her the courage to see him as a contemptuous animal, and the fear of retribution at such boldness had found expression in the condemnatory hallucination of being called a “horse’s ass.”

Now it is such trains of thought—only some parts of which are available to the conscious mind unless one ignores the patient’s objections and presses upon the patient for more details, overcoming resistance through proper intervention—that contain the crux of what clinicians find useful when dealing with patients in the “real world,” and these simply cannot be reliably obtained using structured interviews.
Other psychoanalysts may have made different interpretations of this patient’s hallucinations and arrived at different conclusions. But that only proves the point that the reliability of interviewing dramatically plummets when we go to the deeper layers of the mind and abandon methods of questioning that guarantee beforehand what the answers will be within a narrow range of alternatives.

With regards,
Surendra Kelwala, MD
Livonia, Michigan

Author response
I thank Dr. Kelwala for the thoughtful comments on my article and on my response to Dr. Bennett Cohen’s letter. I also appreciate Dr. Kelwala’s complex and interesting case.

To clarify: I do not maintain that high interrater reliability obtained between interviewers using the Structured Clinical Interview for DSM Disorders (SCID) is “unrelated to the way the questions…[are] structured.” I simply maintain that such structured interviewing methods do not guarantee high interrater agreement; rather, they are “permissive” in so far as they create conditions under which well-trained interviewers may reach agreement. Consider this analogy: Two interviewers have a general notion that being “an American citizen” has to do with whether the person and/or his parents were born in this country. Suppose each interviewer has a half-hour to determine whether the subject is or is not an American citizen. A structured interview that requires questions regarding place of birth is more likely to produce interrater agreement than an interview in which there is no such requirement; and in which the interviewers ask mainly questions about the subject’s taste in music or favorite foods. But there is no guarantee, in the first instance, that the interviewers will reach the same conclusion regarding the subject’s citizenship.

Furthermore, it is not accurate to imply that structured clinical interviews per se “…screen out all forms of psychiatric enquiry where the question cannot be asked in an identical and limited fashion” and “…guarantee beforehand what the answers will be within a narrow range of alternatives.” Much depends on the particular instrument. Some, like CIDI or DIS, which generally use lay interviewers, are designed to elicit only “yes” or “no” responses; moreover, because the interviewers lack clinical interpretation skills, they are not supposed to ask for examples or clarifications from the subjects. None of this is true of the SCID,[1,2] which is the instrument my original article cited in support of good interrater reliability in psychiatry. As explained by Michael B. First MD, one of the developers of the SCID:

“The SCID…is heavily dependent on the clinical judgment of the interviewer in making the ratings. Interviewers are encouraged to ask for examples and to ask extensive follow-up questions for clarification, in the search for the most “valid” answer possible….[for example]…it would be appropriate to ask the kinds of follow-up questions that…differentiate between an auditory hallucination characteristic of a psychotic disorder like schizophrenia from a pseudo-hallucination in dissociative disorders or an intrusive thought in OCD or a mood disorder. While this certainly tends to reduce reliability (as different interviewers may approach the ratings differently), it will improve the overall validity of the diagnosis.” (Personal communication, 1/12/08.)

Moreover, the SCID does not constrain how the patient responds to questions, such as by demanding a “yes” or “no” answer. Interviewers may use “open-ended prompts” to ask for clarification if a patient’s response is ambiguous or equivocal. That is one reason the SCID requires clinical training and basic knowledge of psychopathology.[2]

Finally, I must again stress that “reliability” is distinct from “validity,” or what Dr. Kelwala appropriately describes as the “clinical essence” of a patient’s symptom or condition. Ensuring that a diagnostic process has good interrater reliability is merely the first step toward understanding the “essence” of whatever it is we are investigating (depression, schizophrenia, etc.). I agree with Dr. Kelwala that structured clinical interviews alone are not always the best way of plumbing the depths of the patient’s psyche. I prefer to use a combination of structured questionnaires, such as the Beck Depression Inventory, and a much more open-ended, “existential” discussion with the patient. Sometimes, one learns more from a patient’s poems or habitual metaphors[3] than from structured interviews. But there is no “either/or” choice forced upon us. The use of clinician-based structured interviews helps us achieve greater objectivity in our work. It does not prohibit psychiatrists from using other techniques to probe the “deeper layers of the mind.”

I appreciate the comments of Robert Spitzer, MD, and Michael First, MD, during preparation of this letter.

1. Structured Clinical Interview for DSM Disorders. Available at: Access date: February 6, 2008.
2. Hersen M. Comprehensive Handbook of Psychological Assessment. New York, NY: John Wiley and Sons, 2003:182.
3. Pies R. Metaphor and meaning: How words work magic in our patients. Yale J Humanities Med Sept. 25, 2007. Available at: Access date: February 6, 2008.

With regards,
Ronald Pies, MD
Professor of Psychiatry, SUNY Upstate Medical University, Syracuse, New York; and Clinical Professor of Psychiatry, Tufts USM, Boston, Massachusetts

Acute Dysphoria or Treatment of Emergent Mixed State

Dear Editor:
The November issue of Psychiatry 2007 contained a thoughtful and meticulously detailed discussion of five cases of treatment-emergent ‘dysphoria’ from duloxetine [Sansone R, Sansone L. Duloxetine-related acute dysphoria. Psychiatry (Edgemont) 2007;4(11):65-68]. Could these cases actually represent treatment-emergent mixed states?

After starting duloxetine, the five patients reported the following: “emphatically acknowledged acute and intense dysphoria with irritability;” “never have felt this depressed in my whole life;” “irritability, agitation, depressed like I’ve never felt before;” “a negative spin;” and “irritability, increased depression…the worst I have ever felt.”

The mixed state is particularly miserable because of the presence of BOTH manic symptoms and depressive symptoms. Imagine a person who is deeply depressed (depressed mood, loss of pleasure, hypersomnia, fatigue, psychomotor retardation, and suicidal thoughts) with the sudden addition of dysphoric energy, irritability, sleep disturbance, racing thoughts, inability to focus, an increased desire to get things done, and heightened sexual desire.

Could there be a more miserable state? Is it any wonder that the mixed state predisposes to suicide?

These fine and compassionate physicians listened to their patients, immediately stopped the drug, and the adverse events remitted. I have seen a number of patients with longstanding, unrecognized treatment-emergent mixed states. These patients, in my experience, need anticonvulsant mood stabilizers to stabilize and regularize brain functioning.

With regards,
Susan Delphine Delaney, MD
Plano, Texas