by Norris Turner, PharmD, PhD; Wayne Macfadden, MD; Ravi Anand, MD; Sumant Khanna, MD, PhD; Mark H. Rapaport, MD; J. Thomas Haskins, PhD; Ibrahim Turkoz, MS; and Larry Alphs, MD, PhD  
Drs. Turner and Alphs are employees of Janssen Scientific Affairs, LLC, Titusville, New Jersey; Dr. Macfadden was an employee of Janssen Scientific Affairs, LLC,Titusville, New Jersey, at the time of this analysis; Dr. Anand is a consultant for Anand Pharma Consulting, St. Moritz, Switzerland; Dr. Khanna is a consultant affiliated with The Psychiatric Clinic, New Delhi, India; Dr. Rapaport is Chairman, Psychiatry and Behavioral Sciences, Department of Psychiatry and Behavioral Sciences, Emory University School of Medicine, and Chief of Psychiatric Services, Emory Healthcare System, Atlanta, Georgia; Dr. Haskins was an employee of Johnson & Johnson Pharmaceutical Research and Development, LLC, Titusville, New Jersey, at the time of this analysis; and Mr. Turkoz is an employee of Johnson & Johnson Pharmaceutical Research and Development, LLC, Titusville, New Jersey.

Innov Clin Neurosci 2011;8(10):19–24

Funding: This work was supported by Janssen Scientific Affairs, LLC.

Financial Disclosures: N. Turner and L. Alphs are full-time employees of Janssen Scientific Affairs, LLC, and are Johnson & Johnson stockholders; W. Macfadden was an employee of Janssen Scientific Affairs, LLC, at the time of this analysis; I. Turkoz is an employee of Johnson & Johnson Pharmaceutical Research and Development, LLC, and is a Johnson & Johnson stockholder; J.T. Haskins was an employee of Johnson & Johnson Pharmaceutical Research and Development, LLC, at the time of this analysis; M.H. Rapaport has received consulting and speaking fees and research funds from Janssen Scientific Affairs, LLC, and has served as a consultant for Braincells Inc.; R. Anand has been a paid consultant for Janssen Scientific Affairs, LLC, for various scientific and developmental activities; and S. Khanna has no conflict of interest to disclose.

Key words: bipolar disorder; independent review board, relapse

Abstract: Objective: Independent review boards can provide an objective appraisal of investigators’ decisions and may be useful for determining complex primary outcomes, such as bipolar disorder relapse, in cross-national studies. This article describes the use of an independent, blinded relapse monitoring board to assess the primary outcome (relapse) in an international clinical trial of risperidone long-acting therapy adjunctive to standard-care pharmacotherapy for patients with bipolar disorder. Design: The fully autonomous relapse monitoring board was composed of a chair and two additional members—all psychiatrists and experts in the diagnostic, clinical, and therapeutic management of bipolar disorder. The relapse monitoring board met six times during the study to review patient relapse data and was charged with the responsibility of determining if the events described by investigators qualified as relapses. Additionally, the relapse monitoring board reviewed data for all randomized patients to identify any relapse events not recognized by investigators. Results: Primary efficacy results were similar and significant for investigator- and relapse monitoring board-determined relapses. Ten discrepancies were noted: two of the 42 investigator-determined relapses did not meet the intended clinical relapse threshold as determined by the relapse monitoring board; conversely, the relapse monitoring board confirmed eight relapse events not identified by investigators. The relapse monitoring board had no direct interactions with patients and had to rely on the accuracy of investigator assessments. Also, once an investigator determined a relapse and the patients discontinued the study, less information was available to the relapse monitoring board for relapse assessment.
Conclusions: Use of the relapse monitoring board supported the validity of the study by incorporating a level of standardization to mitigate the risk that local practice in different cultures and medical systems at the sites would confound study results.

Introduction

Independent review boards, such as data and safety monitoring boards, provide autonomous evaluations of safety data to reduce the risk of major adverse health outcomes in clinical trials. This approach can be extended to blinded assessment of treatment response, which traditionally has been determined by clinical trial investigators. An independent review board can provide an objective appraisal to substantiate investigators’ decisions, minimize potential biases and idiosyncrasies associated with individual sites, and extend the applicability of results. Such an approach may be particularly useful for determining complex primary outcomes in cross-national studies.

Criteria for determining bipolar disorder relapse have varied considerably across trials.[1] Some criteria have been based on symptomatic definitions (e.g., as measured by mood rating scales)[2] or syndromal definitions (e.g., as defined by criteria from the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition [DSM-IV]).[3] Other criteria are based on the need for clinical intervention (e.g., new medication, hospitalization, or increased dose of existing medication)[4,5] or study discontinuation due to lack of efficacy.[6] However, each of these methods has limitations. For example, relapse determinations based solely on scales or syndromal definitions are subject to the overvaluation of some symptoms and the omission or understating of others. Additionally, studies with severity-only or intervention-only criteria may identify a broad population, including some patients who did not relapse.

To overcome these limitations, various combinations of relapse criteria have been used.[7,8] However, this approach does not address potential country or cultural biases or the confounding impact of cross-site variability. For example, significant differences have been found between raters’ assessments of manic symptoms in the United States, United Kingdom, and India.[9] Additionally, investigators may intervene to treat patients’ symptoms (e.g., agitation, restlessness, delusions) that preclude significant worsening but would otherwise have met relapse criteria. At times, social issues, rather than clinical concerns, may guide investigator decisions, as in deciding to hospitalize a patient for minor worsening of symptoms if the patient has no family or social support during a weekend.

The objective of this article is to describe the advantages, limitations, and features of a blinded, fully independent, specially constituted relapse monitoring board (RMB) that assessed relapse in an international maintenance trial examining risperidone long-acting therapy (RLAT) adjunctive to standard-care pharmacotherapy in patients with bipolar disorder.[10]

Methods

Study design. The RLAT adjunctive bipolar study—a randomized, double-blind, placebo-controlled, multisite, international trial—was conducted from May 2004 to February 2007 (Study ID: CR004694). Details of its methodology and results have been published elsewhere.10 Sixteen centers in the United States and 16 in India enrolled patients with bipolar disorder Type I or Type II who had had at least four mood episodes that required psychiatric intervention during the past 12 months.

RMB composition. The RMB consisted of a chair and two additional members—all psychiatrists and experts in the diagnostic, clinical, and therapeutic management of bipolar disorder. No RMB members, faculty, or staff at their respective institutions were involved in conducting the clinical trial. The RMB was fully independent, and their compensation was not dependent on study milestones or outcomes.
The initial RMB meeting was convened to obtain agreement on the charter, review the protocol, and establish uniform understanding among its members for the operational definition of a relapse. The RMB met approximately quarterly during the study (a total of 6 times), either in person or through teleconferences, to review principal investigator (PI)-defined relapses and determine whether they constituted relapses according to the RMB (see the following “Relapse Determination” section). The RMB also reviewed data for all of the randomized patients and made independent assessments of relapse. Members participated fully, and consensus was reached on all determinations. Membership did not change during the trial.

Relapse determination. The PIs, blinded to study treatment, conducted an initial assessment of patients to determine whether a relapse occurred based on predefined protocol criteria using established scales, which are outlined in Table 1.[10–14] PIs were required to adhere to the predefined protocol criteria to determine the occurrence and timing of relapses. The RMB was instructed by its charter to use the predefined protocol criteria as a guide (and not a mandate) in determining the occurrence and timing of relapses. RMB relapse adjudication involved confirming whether PI-identified relapses met criteria for bipolar disorder relapse based on these criteria, as well as RMB members’ collective expert judgment. Additionally, the RMB determined whether a relapse not identified by PIs had occurred. The RMB also established, by member consensus, the date of relapses based on blinded data provided by PIs. RMB determinations were used in all primary efficacy statistical analyses.

Information supplied to the RMB for patients with PI-determined relapses included standardized patient narratives; rating scales scores for a patient’s time in the trial; prior and concomitant medications; whether the patient completed the study and, if not, the discontinuation reason; and data from serious adverse events (if present). For patients who did not relapse (as determined by PIs), information supplied to the RMB included clinical database information (i.e., demographic and disease characteristics, rating scales, prior and concomitant medications, study disposition [including reason for discontinuation, if applicable], and all adverse events). Narratives also were developed if requested by the RMB, and the RMB could indirectly query study sites for individual patient data if necessary.

RESULTS

Relapse identification. Among 139 patients (adjunctive RLAT, n=72; placebo, n=67), the RMB identified 48 relapses: 16 (22.2%) patients with RLAT and 32 (47.8%) with placebo. In contrast, PIs identified 42 relapses: 15 (20.8%) with RLAT and 27 (40.3%) with placebo. Based on the assessments of the RMB, RLAT was associated with a significant delay in the time to relapse versus placebo (P=0.004, log-rank test). Time to relapse determined by PIs was also significantly delayed with RLAT versus placebo (P=0.023, log-rank test) (Figure 1). The relative relapse risk, as determined by the RMB, was 2.4-fold (P=0.004, chi-square [Cox regression]) higher with placebo than with RLAT, whereas the risk determined by the PIs was 2.1-fold (P=0.022) higher with placebo than with RLAT.

Relapse discrepancies. Ten discrepancies were found. The RMB determined that two of the 42 PI-determined relapses did not meet the intended clinical threshold for relapse and, therefore, were not counted as relapse events in the calculation of the primary analysis (Table 2). Conversely, eight patients not identified by PIs as having relapsed were determined to have had clinically meaningful relapse events by the RMB (Table 3). The RMB adjudicated such cases as bipolar disorder relapses based on patients’ overall clinical status, longitudinal evidence of mood instability, and rating scale scores. Specifically, one patient withdrew consent because of worsening symptoms but was not identified as having relapsed by the PI; one patient experienced worsening symptoms, which the PI did not consider to be a relapse despite evidence of clinical instability; and one patient experienced symptom worsening that was attributed to treatment nonadherence rather than relapse by the PI. It was unclear why the PIs did not identify relapse in the remaining patients despite mood rating scales that met protocol-defined criteria and other evidence of relapse.

DISCUSSION

To our knowledge, this is the first time an independent review board has been used to determine the primary outcome in an international, multisite study in patients with bipolar disorder. RMB- and PI-determined relapse rates were similar. There was disagreement on 10 relapse determinations. Discrepant relapse determinations reflected the differences in the primary function of the two entities. PIs used protocol criteria but were also guided by dictates of patient care, which may have been influenced by acute needs, social issues, and biases (use of placebo), whereas the RMB determination was based on longitudinal data review, protocol-specific criteria, and clinical expertise.

Unlike the PIs, the RMB did not have personal knowledge of the patients but received sufficient information to make objective decisions. Indeed, the large number of patients reviewed by the RMB resulted in a more complete understanding of unique clinical considerations and factors (e.g., sufficient duration of worsening symptomatology coincident with increased mood symptom rating scales) than PIs might observe. In particular, the RMB highlighted data sets that appeared unusual from a clinical perspective, because of variable responses or changes in scores on either the Young Mania Rating Scale or the Montgomery-Asberg Depression Rating Scale that were inconsistent with global ratings. The RMB thus supported the validity of the study by ensuring that local practice or site-based decisions did not unduly influence study results.
For others who may be considering this approach, the use of such boards complements existing relapse-prevention trial methods and can strengthen study results by providing a means of assessing the reliability of the PI relapse determinations. Of particular note, the RMB was fully independent of the sponsor and was not affiliated with any of the study sites. It contributed international viewpoints of clinical practice standards, therapeutic area expertise, and extensive academic and industry experience. All relapse evaluations were blinded, with longitudinal data, and reflected agreement of all the board members. In addition to protocol-defined rating scale scores, the RMB incorporated an overarching perspective and its collective clinical expertise in relapse determinations. It added further value not only by adjudicating the relapses identified by the PIs but also by assessing all randomized patients to identify cases of relapse that PIs may have missed.  These data suggest that an independent efficacy monitoring board, uninvolved with issues of patient care, may show greater sensitivity in detecting complex outcomes, and should be considered in trials where there is likelihood that involvement in patient care may bias investigator judgment.

The RMB also provided an external independent evaluation of the diagnostic quality of the assessment, rating accuracy and the PIs’ treatment patterns at different centers. An RMB could help identify, for example, any need for additional training for investigators. Potential benefits to be gained by involving an RMB with the study before its initiation include assistance with design questions and alerting the sponsor and research organization to potential issues, thereby improving the overall study quality. Finally, unlike the PIs, the RMB is truly independent and neutral, having neither a financial incentive to recruit patients nor a personal relationship with a patient that may influence relapse decisions.
Several limitations were noted. With no direct patient interaction, the RMB had to rely on the accuracy of the PI assessments. Further, additional time was needed for the study because the RMB was required to confirm all relapse events before the database lock. Finally, once a PI determined the occurrence of a relapse and the patient was discontinued, less information was available to the RMB for relapse assessment.
In conclusion, with PIs guiding patient care and safety, an RMB can provide an independent, reliable, clinical assessment for relapse determination. In conjunction with standardized clinical study methodology, use of an RMB may strengthen study outcomes and may help to establish the study’s clinical uniformity and accuracy and, therefore, the ability to generalize study results across different medical practice patterns and cultures.

Acknowledgments

The authors wish to acknowledge the writing and editing assistance provided by Mariana Ovnic, PhD, Matthew Grzywacz, PhD, and ApotheCom (funding supported by Janssen Scientific Affairs, LLC, Titusville, New Jersey) in the development and submission of this manuscript.

References

1.     Chou JC, Fazzio L. Maintenance treatment of bipolar disorder: Applying research to clinical practice. J Psychiatr Pract. 2006;12:283–299.
2.     Tohen M, Greil W, Calabrese J, et al. Olanzapine versus lithium in the maintenance treatment of bipolar disorder: a 12-month, randomized, double-blind, controlled clinical trial. Am J Psychiatry. 2005;162:1281–1290.
3.     Zarate CA, Tohen M. Double-blind comparison of the continued use of antipsychotic treatment versus its discontinuation in remitted manic patients. Am J Psychiatry. 2004;161:169–171.
4.     Calabrese J R, Suppes T, Bowden CL, et al. A double-blind, placebo-controlled, prophylaxis study of lamotrigine in rapid-cycling bipolar disorder. Lamictal 614 Study Group. J Clin Psychiatry. 2000;61:841–850.
5.     Goodwin GM, Bowden CL, Calabrese JR, et al. A pooled analysis of 2 placebo-controlled 18-month trials of lamotrigine and lithium maintenance in bipolar I disorder. J Clin Psychiatry. 2004;65:432–441.
6.     Keck PE, Calabrese JR, McIntyre RS, et al. Aripiprazole monotherapy for maintenance therapy in bipolar I disorder: a 100-week, double-blind study versus placebo. J Clin Psychiatry. 2007;68:1480–1491.
7.     Tohen M, Calabrese, JR, Sachs GS, et al. Randomized, placebo-controlled trial of olanzapine as maintenance therapy in patients with bipolar I disorder responding to acute treatment with olanzapine. Am J Psychiatry. 2006;163:247–256.
8.     Vieta E, Suppes T, Eggens I, et al. Efficacy and safety of quetiapine in combination with lithium or divalproex for maintenance of patients with bipolar I disorder (international trial 126). J Affect Disord. 2008;109:251–263.
9.     Mackin P, Targum SD, Kalali A, et al. Culture and assessment of manic symptoms. Br J Psychiatry. 2006;189:379–380.
10.     Alphs L, Macfadden W, Haskins JT, et al. A randomized, double-blind, placebo-controlled study of maintenance treatment with adjunctive risperidone long-acting therapy in patients with bipolar I disorder who relapse frequently. Bipolar Disord. 2009;11:827–839.
11.     Spearing MK, Post RM, Leverich GS, et al. Modification of the Clinical Global Impressions (CGI) Scale for use in bipolar illness (BP): the CGI-BP. Psychiatry Res. 1997;73:159–171.
12.     Lindenmayer JP, Czobor P, Alphs L, et al. The InterSePT scale for suicidal thinking reliability and validity. Schizophr Res. 2003;63:161–170.
13.     Montgomery SA, Asberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134:382–389.
14.     Young RC, Biggs JT, Ziegler VE, Meyer DA. A rating scale for mania: reliability, validity and sensitivity. Br J Psychiatry. 1978;133:429–435.