by Leslie Citrome, MD, MPH; Anissa Abi-Dargham, MD; Robert M. Bilder, PhD; Ruth A. Duffy, PhD; Boadie W. Dunlop, MD; Philip D. Harvey, PhD; Diego A. Pizzagalli, PhD; Carol A. Tamminga, MD; Roger S. McIntyre, MD; and John M. Kane, MD

Dr. Citrome is with New York Medical College in Valhalla, New York. Dr. Abi-Dargham is with Stony Brook University in Stony Brook, New York. Dr. Bilder is with the University of California in Los Angeles, California. Dr. Duffy is with Otsuka Pharmaceutical Development and Commercialization in Princeton, New Jersey. Dr. Dunlop is with Emory University in Atlanta, Georgia. Dr. Harvey is with the Miller School of Medicine, University of Miami in Miami, Florida. Dr. Pizzagalli is with Harvard Medical School in Boston, Massachusetts. Dr. Tamminga is with the University of Texas Southwestern in Dallas, Texas. Dr. McIntyre is with the University of Toronto in Toronto, Canada. Dr. Kane is with the Zucker School of Medicine at Hofstra/Northwell in Hempstead, New York.

Funding: The meeting to develop the consensus described here was fully supported by Otsuka Pharmaceutical Development and Commercialization and Lundbeck, and experts received an honoraria for their participation. No payments were made and no financial support or external writing services were provided for the authoring of this manuscript.

Disclosures: Dr. Citrome has served as consultant for AbbVie/Allergan, Acadia, Adamas, Alkermes, Angelini, Astellas, Avanir, Axsome, BioXcel, Boehringer Ingelheim, Cadent Therapeutics, Eisai, Enteris BioPharma, HLS Therapeutics, INmune Bio, Impel, Intra-Cellular Therapies, Janssen, Karuna, Lundbeck, Lyndra, Medavante-ProPhase, Merck, Neurocrine, Novartis, Noven, Otsuka, Ovid, Relmada, Reviva, Sage, Sunovion, Supernus, Teva, University of Arizona, and one-off ad hoc consulting for individuals/entities conducting marketing, commercial, or scientific scoping research; speaker for AbbVie/Allergan, Acadia, Alkermes, Angelini, Eisai, Intra-Cellular Therapies, Janssen, Lundbeck, Neurocrine, Noven, Otsuka, Sage, Sunovion, Takeda, Teva, and Continuing Medical Education (CME) activities organized by medical education companies such as Medscape, NACCME, NEI, Vindico, and Universities and Professional Organizations/Societies; owns stocks (small number of shares of common stock) in Bristol-Myers Squibb, Eli Lilly, Johson & Johnson, Merck, Pfizer purchased over 10 years ago, stock options: Reviva; and received royalties from Wiley (Editor-in-Chief, International Journal of Clinical Practice, through end 2019), UpToDate (reviewer), Springer Healthcare (book), Elsevier (Topic Editor, Psychiatry, Clinical Therapeutics). Dr. Abi-Dargham has received consulting fees and/or honoraria from Sunovion, Otsuka, Merck, and Intracellular Therapies; she holds stock options in Systems 1 Bio and in Terran Biosciences. Dr. Bilder has received consulting fees and/or honoraria from Acadia; Atai Life Sciences; Institute of Digital Media and Child Development, Inc.; Otsuka; ThinkNow, Inc.; and VeraSci. Dr. Dunlop has received research support from Acadia, Compass, Aptinyx, National Institute of Mental Health (NIMH), Sage, and Takeda, and has served as a consultant to Greenwich Biosciences, Myriad Neuroscience, Otsuka, Sage, and Sophren Therapeutics. Dr. Duffy was an employee of Otsuka Development and Commercialization when this project was conducted. Dr. Harvey has received consulting fees or travel reimbursements from Acadia, Alkermes, Bio Excel, Boehringer Ingelheim, Minerva, Regeneron, and Sunovion during the past year; he has signed contracts but received no compensation to date from Karuna and Takeda; he receives royalties from the Brief Assessment of Cognition in Schizophrenia (Owned by Verasci); he is chief scientific officer of i-Function, Inc.; he had a research grant from Takeda and the Stanley Medical Research Foundation. Dr. Pizzagalli has received funding from NIMH, Brain and Behavior Research Foundation, the Dana Foundation, and Millennium Pharmaceuticals; consulting fees from Albright Stonebridge Group, Neumora Therapeutics (former BlackThorn Therapeutics), Boehringer Ingelheim, Compass Pathways, Concert Pharmaceuticals, Engrail Therapeutics, Neurocrine Biosciences, Neuroscience Software, Otsuka, and Takeda; one honorarium from Alkermes; honoraria from the Psychonomic Society for his Editor-in-Chief role of Cognitive, Affective and Behavioral Neuroscience; stock options from Neumora Therapeutics (former BlackThorn Therapeutics), Compass Pathways, Neuroscience Software. Dr. Tamminga has served as an advisor for Karuna Therapeutics and KyNexis, a consultant to Astellas, and Sunovion and serves on a Merck Data Safety Monitoring Board (DSMB), in addition to holding Karuna stock options. Dr. McIntyre has received research grant support from Canadian Institutes of Health Research (CIHR)/ Global Alliance for Chronic Diseases (GACD)/Chinese National Natural Research Foundation; he has received speaker/consultation fees from Lundbeck, Janssen, Purdue, Pfizer, Otsuka, Allergan, Takeda, Neurocrine, Sunovion, Bausch Health, Novo Nordisk, Kris Eisai, Minerva, Intra-Cellular, Abbvie; he is CEO of Champignon. Dr. Kane has received consulting fees and/or honoraria for lectures from Alkermes, Dainippon Sumitomo, Intracellular Therapies, Janssen, LB Pharma, Lundbeck, Merck, Neurocrine, Otsuka, Roche, Saladex, Sunovion and Teva; he is a shareholder in LB Pharma and The Vanguard Research Group.

Innov Clin Neurosci. 2022;19(1–3):26–32.

ABSTRACT: The Research Domain Criteria (RDoC) initiative aims to organize research according to domains of brain function. Dysfunction within these domains leads to psychopathology that is classically measured with rating scales. Examining the correspondence between the specific measures assessed within rating scales and RDoC domains is necessary to assess the needs for new RDoC-focused scales. Such RDoC-focused scales have the potential of allowing translation of this work into the clinical domain of measuring psychopathology and designing treatment. Here, we describe an initial qualitative assessment by a group of 10 clinician-scientists of the alignment between RDoC domains and the items within five commonly used rating scales. In this commentary, we report limited correspondence and make recommendations for future work needed to address these limitations.

Keywords: Research domain criteria, mental disorder, depression, anxiety, psychosis, rating scales

In 2009 the United States (US) National Institute of Mental Health (NIMH) launched the Research Domain Criteria (RDoC) initiative to provide a new and transformative framework for orienting research on mental disorders.1 Unlike most conventional nosological systems that focus on syndromal classification based on symptoms, such as the Diagnostic and Statistical Manual (DSM) or International Classification of Diseases (ICD), the RDoC prioritizes basic dimensions of functioning that span the full range of human behavior, from normal to abnormal, and proposes that these dimensions be probed across units of analyses (e.g., molecules, cells, circuits, physiology, observable behavior, and self-reported experiences). The RDoC strategy includes molecular, cellular, circuit, and behavioral functions and identifies precise probes associated with abnormal behavior, promising to lead to a better understanding of the biological processes that underlie mental illnesses and provide the basis for more targeted treatments. The conventional diagnostic systems (DSM, ICD) have improved reliability of diagnoses, but their major limitations persist, including significant overlap of syndromes that creates apparent “comorbidity,” substantial heterogeneity, and lack of validity with respect to biological correlates, pathophysiology, or direction for novel treatment development. Despite these limitations, the conventional diagnoses continue to guide treatment recommendations and form the basis of decision-making and trial designs underlying approval of new treatments by medical regulatory agencies, though there is an increasing openness to considering RDoC-related symptom constructs in labeling.2

While RDoC’s immediate goals are to deepen the understanding of neurobiological underpinnings of psychiatric disorders, the ultimate goal is to inform and transform therapeutic developments for mental disorders.3 The ideal outcome to emerge from this initiative would be to have measurable, precise alterations across an RDoC domain or subdomain that can be specifically addressed with targeted interventions. These interventions would likely differ across individuals who currently share the same diagnostic category per DSM. Thus, fundamentally, the progress envisioned by RDoC is closely related to the vision for precision medicine promoted by the US National Institutes of Health (NIH).4

An interesting feature of RDoC is that among its units of analysis, the symptoms most critical to conventional diagnosis are not explicitly included. Self-reports might include symptom rating scales, but in general, this unit of analysis so far has not included the criterion symptoms in the DSM nor the instruments that are used to assess these symptoms in mental health research. While progress is being made for evaluating the components of the matrix of RDoC in other units of analysis, one area that merits attention is the level of correspondence between the diagnosis-focused clinical scales used to measure therapeutic benefit and the domains of function studied under RDoC. Decades of DSM-based research using well-established symptom scales has produced large clinical datasets, with associated biological data that could be mined to yield new insights if the rating scales can be aligned with RDoC constructs. How well do the most commonly used rating scales, for which comparative changes lead to the approval of new therapies, serve the purpose of measuring RDoC constructs and response to interventions based on the RDoC framework? The six functional domains, as currently defined by RDoC, are Negative Valence Systems (NVS), Positive Valence Systems (PVS), Cognitive Systems (CS), Social Processes (SP), Arousal and Regulatory Systems (ARS), and Sensorimotor Systems (SS),5 are agnostic to DSM diagnosis and not targeted at symptoms; thus, it is possible that current rating scales will not be useful for assessing behavior and symptoms within the RDoC domains most relevant to specific diseases. In this regard, it is possible that they would require reconfiguration and reorganization according to RDoC domains (for an initial example, see Khazanov et al6). Understanding this correspondence, or lack thereof, is critical for moving forward and, eventually, best applying the benefits of the RDoC initiative to inform the design of clinical trials and ultimately impact on clinical care. At that point in time, a new characterization of therapeutic response, with new rating scales rooted in RDoC domains or subdomains, and new FDA guidance will be needed. In the meantime, however, information providing an initial proof of concept might be obtained from exploratory re-analyses of currently available clinical trials data considering domain specific subsets of rating scales.

In pursuit of this goal, this commentary describes the process and outcomes of an effort to examine the correspondence between RDoC domains and elements of commonly used clinical rating scales by a group of 10 experts with clinical and research expertise across different disease areas and experimental methodologies. These experts come from varying academic and research backgrounds and have a range of preclinical and clinical experience, including psychopathology, pharmacology, clinical trials, imaging, neuropsychological and biomarker evaluation, psychosocial and workplace functioning, and social cognition. All have a demonstrated interest in the utilization of the theoretical framework examining domains of behavior within their areas of research, as evidenced by their ongoing studies and their prior publications, such as examining predictors for  response to antidepressant medication.7,8

Methods

A group of 10 researchers (LC, AAD, RMB, BWD, PDH, DAP, CAT, RSM, JMK and Madhukar Trivedi) were tasked with assigning over 100 line-items taken from a set of seven rating scales frequently used in clinical trials for depression and psychosis to one of the six RDoC domains. The following rating scales were selected: Montgomery-Asberg Depression Rating Scale (MADRS);9 17-item Hamilton Depression Rating Scale (HAM-D17);10 Inventory of Depressive Symptomatology, Self-Report (IDS-SR);11 Hamilton Anxiety Rating Scale (HAM-A);12 Sheehan Disability Scale (SDS);13 Positive and Negative Syndrome Scale (PANSS);14,15 and the Personal and Social Performance (PSP) Scale.16 The group completed their assignment through a two-stage process in which they first assigned, via a digital platform (Radius Direct), each item of these rating scales to a specific RDoC domain. The instructions were to attempt to assign a single domain or subdomain to each item in the scales, although the online platform allowed participants to assign an item to more than one domain. Initial consensus was calculated based on the online feedback. As participants placed some items in more than one domain, consensus could not be defined only by the greatest percentage of votes out of 10 participants. Therefore, the initial definition of consensus was operationalized as all of the following: a selection made by at least five of the 10 participants, a selection that received 40 percent or more of the total votes, and a selection had to be separated from the next most frequent domain selection by more than 10 percent of the votes and had to have fewer than five votes for that next most frequent domain selection.

This process was then followed by a live meeting to reach consensus on assigning all items. As part of that discussion, it was agreed that two of the scales, the SDS and the PSP, should be removed from consideration because they evaluate functional performance, rather than psychopathology, and have limited validity and utility, particularly when measured over short time periods.17 In addition, the group felt individual items of the SDS could be allocated across multiple RDoC domains, thus lacking specificity. Moreover, self-reports of everyday functioning across samples of participants meeting criteria for different DSM diagnoses have been shown to have limited correlation with objective reports or performance-based assessments of functionality and to be influenced by current mood states.18–22 Finally, the PSP does not provide clear guidance on how information should be obtained for ratings, thus leading to a situation where similar scores on the PSP could originate from completely different information sources.23

Results

When the initial online results were tabulated using the consensus definition and following the removal of the SDS and PSP from the task (7 items), the group was able to reach initial consensus through the online survey for 42 of the remaining 101 items (Table 1) that spanned across the five remaining scales (MADRS, HAM-D17, IDS-SR, HAM-A, and PANSS).

The best example of consensus was reached for the RDoC domain of NVS, where the group was able to place select items from the five scales related to depressed or sad mood and anxiety. For CS, the group reached the consensus definition for items from the five scales, with the predominant items being from the PANSS across the positive, negative, and general psychopathology subscales related to judgement, conceptual organization, and abstract thinking. Both the ARS and SS domains were assigned items from the five scales mostly related to the sleep/wake cycle and general arousal. However, for both PVS and SS there was more variability in the initial online voting, leading to discussions during the consensus meeting. At that meeting, the group reached consensus on the primary RDoC domain for 50 additional scale items, for a total of 92 items from the five remaining scales (Table 2). The cognition and arousal RDoC domains had the greatest number of scale items reaching consensus, and each accounted for over 20 percent of the total items, with less than 10 percent of the items placed in each of the sensorimotor and social domains.

During the live discussion, the group identified three types of challenges that arose in matching scale items to RDoC domains, categorized into three sections: 1) Items that fit into multiple domains; 2) Items that could not be assigned to any domain (general somatic symptoms); and 3) “Difficult to classify” items.

Items that fit into multiple domains. Most items could not be assigned to a single domain, as they informed multiple domains. While the consensus definition was reached in the online voting for 41.6 percent of the items, upon meeting live, the group felt there was justification for more than one RDoC domain for many of the items because of the breadth of their definition. For this reason, the group decided most items should be assigned to both a primary and a secondary domain. For example, MADRS Item 2, reported sadness, contains elements of both NVS and PVS domains in its description. The description states, “representing reports of depressed mood, regardless of whether it is reflected in appearance or not.” It includes low spirits, despondency, or the feeling of being beyond help without hope, which justifies classification within the NVS, consisting of loss, sustained threat, and frustrative nonreward. The description also recommends rating the item according to intensity, duration, and the extent to which the mood is reportedly influenced by events, which fits the description of PVS, including reward responsiveness and reward valuation.24

RDoC domains and line items were evaluated as written; thus, an additional challenge in parsing out the items was the interpretation of an item’s description. The group agreed to interpret the intent of each line item based on its description as written to avoid assumptions regarding the etiologies, biological or psychological, of the symptom. Assignment of the items could be influenced by how the measurement is made, whether by subjective report or objective assessment. For example, the NVS domain appears to be predominantly involved when the scale requires an individual to give a subjective report regarding their mental experience. In contrast, an objective assessment by a rater or clinician might not categorize those symptoms as part of the NVS domain and, as noted in the example above, the objective rater assessment would yield a different result.

Moreover, similar line items from separate scales were assigned to different domains, as their specific descriptions differed. Because the group interpreted the item descriptions as written, many items that superficially appeared similar across scales were assigned to different RDoC domains, suggesting variability in the detailed descriptions of a specific symptom across scales. Some examples of similar-appearing items leading to diverging RDoC domain assignments were tension, sadness, psychomotor retardation, activity, guilt, insight, self-perception, social withdrawal, and thought disorder. Table 3 outlines the details regarding these items. These examples, and the others described earlier, illustrate the challenges in assigning rating scale items to single RDoC domains and subdomains. Development of new scales that align more closely with RDoC domains might be necessary in the future to apply the concepts from RDoC into clinical applications.

Items that could not be assigned to any domain (general somatic symptoms). General somatic symptoms need to be integrated into the RDoC matrix; however, the framework needs to be further developed to incorporate these items. Upon discussion, the group agreed that items in the HAM-D17, IDS-SR, and HAM-A describing general somatic symptoms did not fit into the current RDoC framework. The inability to make RDoC classifications of these symptoms commonly experienced across DSM disorders led the group to consider the value of adding an interoception domain to the RDoC matrix. This new domain would encompass self-monitoring and self-perception of somatic sensations, such as pain and other symptoms. Substantial research supports the neurobiology of interoceptive processing, disruptions of which might be relevant across psychiatric illnesses and treatments.24,25

Additional components that were suggested for this domain included self-assessment of the quality of actions and decisions, social cognitive understanding of scenarios, and the ability to learn and perform skilled acts, a critical deficit in many severe mental illnesses.

“Difficult to classify” items. Many items were challenging to classify into RDoC categories. These items remain highly clinically relevant. For example, items related to suicide, such as the HAM-D Item 3, suicide, include a wide span of symptoms, from anticipatory affect, cognitive ideation to impulsivity and actual suicide attempts, which involve taking an action. NVS, CS, and PVS were all considered by the group without reaching an agreement. Similar difficulty was encountered for the MADRS pessimistic thoughts item. Experts thought it could be allocated to NVS, PVS, CS, and SP if extreme dysfunctions occur within any of these domains. Additional items for which consensus could not be reached included the IDS-SR items feeling irritable, quality of your mood, view of my future, and general interest.

Several PANSS items also yielded difficulties in classification. For the PANSS Item P1, delusions, CS was selected as the primary domain because of the item’s description of cognitive distortions. SP was selected as the secondary domain because the item description evaluated the effect of delusions on social interactions and behavior. For the PANSS Item G8, uncooperativeness, advisors selected SP as the primary domain but could not agree on a secondary domain because the description was too broad. For the PANSS Item G14, poor impulse control, CS and SS were selected as co-primary domains because the line item’s description contains mention of cognitive control, executive motor planning, and motor inhibition of volitional activity. ARS was selected as the secondary domain because the description contains elements of the hair-trigger response. The PANSS Item P7, hostility, was assigned to SP as primary domain because the description contains elements of social communication. Advisors agreed that the basis for rating was observed behavior as opposed to patient perception of their behavior. Additional potential domains for this item included NVS, PVS, and CS. Item N2, emotional withdrawal, was assigned to PVS as the primary domain because the item’s description contained elements of anhedonia. SP was selected as the secondary domain.

Discussion

To the best of our knowledge, this is the first attempt at linking RDoC to commonly used clinical rating scales. Here, we performed a qualitative review of the items from five widely used clinical scales to examine how well they correspond to RDoC domains. We showed that rating scales have limitations in their applicability to RDoC domains. This is not surprising, considering that, fundamentally, the RDoC is agnostic to the phenomenologically based DSM definitions of mental disorders and the similarly anchored clinical rating scales currently in use.

We noted that some rating scale items fit into multiple domains. It is possible that the same behavioral manifestation can relate to multiple domains due to an actual overlap in brain mechanisms, leading to similar behavioral expression reflected in one item. For example, there is significant overlap between CS and SS regarding cognitive control, executive motor planning, and motor inhibition. Another potential explanation is a deficiency in the description of the items themselves, rendering them imprecise or ambiguous; revision of the item descriptors and/or basis for ratings could, however, negatively alter the scale’s psychometric properties.

Another observation is that some items of high clinical importance, such as suicide, did not fit into any RDoC domain. Similarly, self-destructive behavior, impulsivity, and risk taking were difficult to classify. These are important dimensions of behavior that should be captured within the RDoC matrix if it is to become a tool to guide clinical evaluation. This raises the question of whether additional RDoC constructs should be developed to obtain information on clinical aspects that are presently not represented. A similar consideration applies to somatic symptoms and interoception.

As noted above, RDoC does not capture self-perception and response biases, which could be linked to a variety of functional outcomes and might also be implicated in suicidal ideation and behavior.

A question that arises from this exercise is whether further development of the RDoC framework would benefit from the creation of new rating scales that are specific to discrete domains, avoiding overlap between domains. This could start by regrouping items from existing scales based on a statistical assessment of which items relate best to which domain. Some work is already underway. A group of researchers developed a PVS scale of 21 items measuring responses to a wide range of rewards, including food, physical touch, being outdoors, positive feedback, social interactions, hobbies, and goals.6 This scale showed good validity, factorial reliability, and internal consistency and was better related to reward than negative valence, depression, or anxiety. This type of work should extend to other domains and subdomains. Applying natural language processing to questionnaire data using word embeddings is another approach that might enable use of existing questionnaire datasets to address RDoC approaches to mental illness.26

Limitations. A limitation of this exercise was that scale items were linked only to the level of RDoC domain, even though each domain subsumes several constructs. For example, NVS includes anxiety, fear, and loss, each of which has been linked to separate biological features, though our classifications of the scale items linked only to the domain level. A further limitation of the scales, which is tacit to the limitations of RDoC, is that the scales do not have the ability to consider potential changes in phenomenology across the illness trajectory.27,28

Conclusion

The RDoC is a novel approach to understanding brain function and dysfunction that leads to abnormal symptoms and behavior. Our main recommendations include quantitative testing of the correspondence between rating scale items and RDoC subdomains using statistical methods, followed by development of more specific rating scales to capture RDoC subdomains and examination of therapeutic efficacy of agents available and in development with these new scales by using large data sets derived from clinical trials. Better alignment of the transformative RDoC initiative with contemporary clinical trial methodology will benefit both fields and ultimately translate into a larger impact on optimizing health for our most vulnerable patients.

References

  1. Insel T, Cuthbert B, Garvey M, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry. 2010;167(7):748–751.
  2. Sanislow CA, Ferrante M, Pacheco J, et al. Advancing translational research using NIMH research domain criteria and computational methods. Neuron. 2019;101(5):779–782.
  3. Sanislow CA. RDoC at 10: changing the discourse for psychopathology. World Psychiatry. 2020;19(3):311–312.
  4. Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372(9):793–795.
  5. National Institute of Mental Health. RDoC matrix. https://www.nimh.nih.gov/research/research-funded-by-nimh/rdoc/constructs/rdoc-matrix. Accessed 19 Jan 2022.
  6. Khazanov GK, Ruscio AM, Forbes CN. The positive valence systems scale: development and validation. Assessment. 2020;27(5):1045–1069.
  7. Ang YS, Kaiser R, Deckersbach T, et al. Pretreatment reward sensitivity and frontostriatal resting-state functional connectivity are associated with response to bupropion after sertraline nonresponse. Biol Psychiatry. 2020;88(8):657–667.
  8. Ahmed AT, Frye MA, Rush AJ, et al. Mapping depression rating scale phenotypes onto research domain criteria (RDoC) to inform biological research in mood disorders. J Affect Disord. 2018;238:1–7.
  9. Montgomery SA, Asberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134:382–389.
  10. Khan A, Khan SR, Shankles EB, Polissar NL. Relative sensitivity of the Montgomery- Asberg Depression Rating Scale, the Hamilton Depression rating scale and the Clinical Global Impressions rating scale in antidepressant clinical trials. Int Clin Psychopharmacol. 2002;17(6):281–285.
  11. Rush AJ, Giles DE, Schlesser MA, et al. The Inventory for Depressive Symptomatology (IDS): preliminary findings. Psychiatry Res. 1986;18(1):65–87.
  12. Hamilton M. The assessment of anxiety states by rating. Br J Med Psychol. 1959;32(1):50–55.
  13. Leon AC, Olfson M, Portera L, Farber L, Sheehan DV. Assessing psychiatric impairment in primary care with the Sheehan Disability Scale. Int J Psychiatry Med. 1997;27(2):93–105.
  14. Kay SR, Fiszbein A, Opler LA. The Positive and Negative Syndrome scale (PANSS) for schizophrenia. Schizophrenia Bull. 1987;13:261–276.
  15. Kay SR, Opler LA, Fiszbein A. Positive and Negative Syndrome Scale (PANSS) Technical Manual. MultiHealth Systems, Inc.: Toronto, Ontario; 2006. 
  16. Morosini PL, Magliano L, Brambilla L, et al. Development, reliability and acceptability of a new version of the DSM-IV Social and Occupational Functioning Assessment Scale (SOFAS) to assess routine social functioning. Acta Psychiatr Scand. 2000;101(4):323–329.
  17. Weiller E, Weiss C, Watling CP, et al. Functioning outcomes with adjunctive treatments for major depressive disorder: a systematic review of randomized placebo-controlled studies. Neuropsychiatr Dis Treat. 2018;14:103–115.
  18. Kaye JL, Dunlop BW, Iosifescu DV, et al. Cognition, functional capacity, and self-reported disability in women with posttraumatic stress disorder: examining the convergence of performance-based measures and self-reports. J Psychiatr Res. 2014;57:51–57.
  19. Strassnig M, Kotov R, Fochtmann L, et al. Associations of independent living and labor force participation with impairment indicators in schizophrenia and bipolar disorder at 20-year follow-up. Schizophr Res. 2018;197:150–155.
  20. Yoo-Jeong M, Anderson A, Rahman AF, et al. Associations of mood on objective and subjective cognitive complaints in persons living with HIV/AIDS. J HIV AIDS. 2018;4(1): 10.16966/2380-5536.146.
  21. Strober LB, Binder A, Nikelshpur OM, et al. The perceived deficits questionnaire: perception, deficit, or distress? Int J MS Care. 2016;18(4):183–190.
  22. Sumiyoshi T, Watanabe K, Noto S, et al. Relationship of cognitive impairment with depressive symptoms and psychosocial function in patients with major depressive disorder: cross-sectional analysis of baseline data from PERFORM-J. J Affect Disord. 2019;258:172–178.
  23. Schaub D, Brune M, Bierhoff HW, Juckel G. Comparison of self- and clinician’s ratings of personal and social performance in patients with schizophrenia: the role of insight. Psychopathology. 2012;45(2):109–116.
  24. Khalsa SS, Adolphs R, Cameron OG, et al. Interoception and mental health: a roadmap. Biol Psychiatry Cogn Neurosci Neuroimaging. 2018;3(6):501–513.
  25. Khoury NM, Lutz J, Schuman-Olivier Z. Interoception in psychiatric disorders: a review of randomized, controlled trials with interoception-based interventions. Harv Rev Psychiatry. 2018;26(5):250–263.
  26. Sonabend WA, Pellegrini AM, Chan S, et al. Integrating questionnaire measures for transdiagnostic psychiatric phenotyping using word2vec. PLoS One. 2020;15(4):e0230663.
  27. McIntyre RS. In vivo phenotyping, mechanism-informed treatments, domain-based psychopathology and nomological networks: a strategy for treatment discovery and development in bipolar depression. Bipolar Disord. 2020;22(7):657–659.
  28. Tai AMY, Albuquerque A, Carmona NE, et al. Machine learning and big data: implications for disease modeling and therapeutic discovery in psychiatry. Artif Intell Med. 2019;99:101704.