Implementation of the Electronic Columbia-Suicide Severity Rating Scale (eC-SSRS™) Across Four Phase IIIB Clinical Trials in HIV-Infected Individuals (ARIA, STRIIVING, DAWNING and INSPIRING)

| July 1, 2018 | 0 Comments

By Clare Brennan, DPT, MSPH; Hannah Whillis, MS; Choy Man, BS; Brian Wynne, MD; Vani Vannappagari, MBBS, MPH, PhD

Dr. Brennan and and Ms. Man are with Clinical Development and Dr. Vannappagari is with Epidemiology and Real World Evidence at ViiV Healthcare in Research Triangle Park, North Carolina. Ms. Whillis was with GlaxoSmithKline, plc, in London, England, at the time of this work. Dr. Wynne is with Global Medical Affairs at ViiV Healthcare in Collegeville, Pennsylvania.

Funding: This study was funded by ViiV Healthcare.

Disclosures: Drs. Brennan, Wynne, and Vannappagari and Ms. Man are employees of ViiV Healthcare and own stock in GlaxoSmithKline. Ms. Whillis was an employee of GlaxoSmithKline at the time of this work.

Abstract: Objective: Increased rates of suicidal ideation/ behavior have been reported in individuals with human immunodeficiency virus infection/acquired immunodeficiency syndrome. The electronic Columbia-Suicidality Severity Rating Scale (eCSSRS <sup>™</sup>) is a validated tool for assessment of suicidal risk. The objective of this study was to assess the site perspectives on implementation of the eC-SSRS used in Phase IIIb studies of dolutegravir. Methods: We developed and validated the ViiV eC-SSRS Metrics and Perspectives Site Questionnaire (VEQ). Topics included ease of eC-SSRS administration, agreement with clinical assessment, unreported risk, and confidence in utility of the eC-SSRS. Results: Clinical data from two Phase IIIb studies were reviewed for correlation with the eC-SSRS results. The overall VEQ response rate was 83%. A total of 85% of respondents administered the eC-SSRS by phone, and 34% reported their patients would be unable to complete a web survey. First-time eC-SSRS users made up 64% of the responders; 85% of repeat administrators said implementation became easier over time. One-half said the eC-SSRS accurately predicted risk, and 14% said the eC-SSRS identified previously unreported risk. A total of 65% were somewhat/very confident their patients are being assessed accurately for suicide risk. Conclusion: Results of the eC-SSRS from ARIA and STRIIVING were consistent with the clinical data. The eC-SSRS identified previous unreported risk for suicidality and provided physicians opportunity for follow-up. Respondents felt the eC-SSRS helps them manage suicide risk, and they reported that its administration became easier with experience. Thus, the eC-SSRS is considered a useful tool in this setting.

Keywords: dolutegravir, HIV-1, integrase inhibitor, eC-SSRS, suicide risk assessment

Innov Clin Neurosci. 2018;15(7–8):15–19

Apart from human immunodeficiency virus (HIV) infection/acquired immunodeficiency syndrome (AIDS)-related causes, suicide is one of the most frequent causes of death among people living with HIV (PLHIV).[1] The rate of suicide among PLHIV is higher than the general population.[2] In the Swiss HIV Cohort Study, suicide was the cause of death among three percent (1988–1995, 1996–2004), and six percent (2005–2010) of individuals (n=5,023).[1] Among studies in the highly active antiretroviral therapy (ART) era, rates of deliberate self-harm and suicidal ideation averaged 19.9 percent (range 0–62%) and 26.9 percent (range 3.9–78%), respectively.[3]

Draft industry guidance from the United States Food and Drug Administration (FDA) endorses prospective assessments of treatment-emergent suicidal ideation/ behavior in clinical trials of patient populations at risk or drugs/disease with risk.[4] The Columbia-Suicide Severity Rating Scale (C-SSRS) is implemented in mental healthcare and research settings to assess the degree of suicidal ideation and to predict suicidal behavior in at-risk individuals.[5] The C-SSRS is available as a provider- or researcher-administered survey or as an electronic survey (eC-SSRS) administered via phone, computer, or tablet. The C-SSRS adheres to the FDA recommendation for classifying suicidal ideation and behavior into 11 categories, and it is cited by the FDA as a useful and validated instrument for assessing suicidal ideation and behavior in industry-led clinical trials.[4]

Currently, suicidal ideation/behavior among participants is monitored in all ViiV Healthcare-sponsored interventional HIV trials using the eC-SSRS at every clinical visit during the study period, allowing an investigator to take prompt and appropriate action if a participant is identified to be at increased risk.

Although the validity of the C-SSRS has been assessed, a paucity of research has been conducted regarding the administration of the instrument and its acceptability among providers, research personnel, and patients. From the perspective of survey interviewers, Giddens and colleagues[6] argue that the C-SSRS not only fails to capture all 11 FDA categories of suicidal ideation and behavior, but that it also contains confusing skip logic that does not accommodate all potential response combinations. This shortcoming compels interviewers to follow the most appropriate sequence of questions for a given respondent, resulting in incorrect responses and unintentional bias. Additionally, the wording of some of the survey items is ambiguous and compels the interviewer to interpret the question, potentially compromising the validity of the instrument.[6]

From the patient perspective, Mundt and colleagues[7] administered a verbal C-SSRS and phone-based eC-SSRS to 20 individuals (10 inpatient psychiatric patients and 10 controls) and solicited feedback on the acceptability of each version. Several participants reported both versions were equally easy to understand and respond to. Four of the controls preferred the phone-based eC-SSRS, whereas the remaining six reported no preference. Respondents who preferred the phone-based eC-SSRS reported the privacy of the phone and lack of interviewer bias as benefits, whereas those who preferred the verbal C-SSRS reported the ability to ask a human (vs. an automated system) clarifying questions as a benefit.


The Phase IIIb studies of dolutegravir employed suicidality screening using the eC-SSRS to assess treatment-emergent suicidality. The C-SSRS gathers history on an individual’s suicidal behavior and intensity of suicidal ideation at baseline (to ascertain lifetime and current experiences) and at each subsequent clinical study visit (to ascertain suicidal ideation/behavior since the last study visit). At the beginning of each study visit, participants complete the eC-SSRS. If suicidal ideation/behavior is identified, then an alert is sent to the site at that visit to prompt further evaluation. A false/unconfirmed alert is one where the site determines the participant does not have suicidal risk and/or he or she mistakenly provided a wrong answer (e.g., pressed wrong button) when completing the eC-SSRS.

To evaluate the ease of administration and utility of the eC-SSRS as perceived by the site staff, ViiV Healthcare (Research Triangle Park, North Carolina) developed and administered the ViiV eC-SSRS Perspectives Questionnaire (VEQ). The main objectives of the VEQ were to assess site perceptions on implementation of the eC-SSRS, the general acceptability of the eC-SSRS among providers and research personnel, concordance between the eC-SSRS and independent assessment by clinicians of past suicidality, and the value of the tool in relation to the Phase IIIb clinical study experience.



We developed the VEQ to include questions on the mode of eC-SSRS administration (phone vs. internet), first-time site use of eC-SSRS, ease of administration of the eC-SSRS, agreement with the clinical assessment, identification of unreported suicidality, and confidence in the utility of the eC-SSRS (Appendix 1). The VEQ was distributed via External Select SurveyMonkey® from June 5, 2015, through July 17, 2015, in 14 countries at 218 sites across four clinical studies investigating dolutegravir for the management of HIV-1 infection (Table 1). The four clinical studies are as follows (described in Table 2):


The Phase IIIb studies of dolutegravir

TABLE 2. VEQ completion rate by study.

Each site was asked to have one individual, either the investigator or the site study coordinator, complete the VEQ. Site personnel active in more than one of the four clinical studies were asked to complete the VEQ to summarize their experience with the eC-SSRS during participation in each study; thus, some site members completed more than one VEQ. During the survey period, reminder emails were sent to non-responding sites to boost the response rate.


The number of sites that completed the VEQ are shown by country in Table 1 and by study in Table 2.

Of the VEQ respondents, 27 percent were investigators, 67 percent were study coordinators, and 11 percent were “other” (e.g., subinvestigators, data managers, research directors). First-time administrators of the eC-SSRS made up the majority (64%) of the VEQ respondents; 85 percent of repeat administrators said implementation of the eC-SSRS became easier over time.

Most respondents administered the eC-SSRS via phone, with smaller proportions using the web/internet or both the phone and web to administer the eC-SSRS to study participants (Tables 3 and 4).

TABLE 3. eC-SSRS version and ease of use

TABLE 4. Proportion of subjects with eC-SSRS agreement with clinical assessment

Approximately two-thirds of respondents reported their patients would successfully complete a web-based survey if the eC-SSRS phone survey was unavailable, whereas one-third said their patients would not be successful in completing a web-based survey.

When queried about the ease/difficulty of administering the eC-SSRS to the participants (how easy it was to explain the reason for the survey and how to take the survey, considering any other help the site had to provide), most said it was very easy or easy, and relatively few said the tool was hard or very hard to administer (Table 3). Of the respondents saying hard or very hard, more than two-thirds were first-time users, and more than three-fourths used the phone to administer the eC-SSRS. Among the respondents using the phone version, few reported that the phone version was hard or very hard to administer, and, among those using the web version, none reported that administration of the eC-SSRS was hard or very hard.

Next, we determined if results from the eC-SSRS correlated with the investigator’s clinical assessment of the participant at the visit during which the eC-SSRS was completed. Most respondents said the eC-SSRS result agreed with the clinical assessment in 90 percent or more of patients, whereas relatively few reported the eC-SSRS agreed with the clinical assessment less than 50 percent of time (Table 4). Of these respondents, 7 of 16 were investigators, 8 of 16 were from the ARIA study, and 10 of 16 reported the eC-SSRS overestimated the risk for suicidality.

Overall, one-half of respondents said the eC-SSRS accurately assessed the risk for suicidality, with 14 percent saying that the eC-SSRS identified a previously unreported risk (Figure 1). In addition, approximately one-half of the respondents who reported that the eC-SSRS overestimated the risk for suicidality were investigators.

FIGURE 1. Agreement level between eC-SSRS and subsequent clinician assessment of suicidality (Note: Sites could select >1 response)

The VEQ also examined reasons patients refused to complete the eC-SSRS assessment and rate of refusal. A total of 95 percent of respondents said less than 25 percent of patients refused to complete the eC-SSRS assessment. Reasons for refusal included privacy/confidentiality, too time consuming, and other reasons such as “waste of time,” “afraid it will disqualify them,” “hate this survey,” and “the survey itself is depressing.”

For site staff to administer the eCSSRS without any potential bias to study participants, it is important for the individuals administering the eC-SSRS to have confidence in the utility of the instrument. The VEQ showed that 65 percent of respondents were very/somewhat confident their patients were being assessed accurately for suicidal risk and that the eC-SSRS helps the site manage this risk appropriately.

We determined the value of the eC-SSRS in relation to the Phase IIIb clinical study experience in two of the four completed studies with data available at the time of this writing. At Week 48 in ARIA (a study in HIV treatment-naïve women) and STRIIVING (a study in virologically suppressed adults), we evaluated if any cases of suicidal ideation/behavior were identified by the eC-SSRS and not by clinical assessment. Both studies used the phone version of the eC-SSRS.


The ARIA study[8] enrolled 499 women with HIV-1 infection; 2,806 (705 baseline) eC-SSRS assessments had been conducted at the time of completion of the VEQ. The STRIIVING study[9] enrolled 553 men and women; 4,352 (841 baseline) eC-SSRS assessments had been conducted at the time of VEQ completion. Of the 1,052 participants in the two studies, at baseline, there were 111 patients with alerts; 99 (89%) were considered true positive alerts by the investigators and 12 (11%) were false/ unconfirmed alerts, indicating an overall prevalence of true positive alerts of nine percent. One patient in ARIA was excluded from study participation due to a positive eC-SSRS result at the randomization screening visit. Of the 99 patients with a true positive alert at baseline, four (4%) went on to have a post-baseline alert. Of the remaining 953 patients without a true positive baseline alert, six (<1%) had a true positive alert post-baseline. While the studies were being conducted, 37 patients had 38 positive alerts: 10 (29%) had true positive alerts and 27 (71%) had false/unconfirmed alerts.

In STRIIVING, one participant attempted suicide and one reported suicidal ideation.[9] In ARIA, one participant reported intentional self-injury and seven participants reported suicidal ideation (1 participant with 2 positive e-CSSRS alerts).[8] All 10 patients with true positive alerts post-baseline also had a suicidality-related adverse event (AE)/serious AE documented in the electronic case report form. One suicidality-related AE was detected during the clinical assessment but not by the eC-SSRS. The main reasons for false-positive alerts were due to patients pressing the incorrect button on their phone or responding with their previous (lifetime) history of suicidality during a post-baseline assessment.


Despite new ARTs and improved survival rates and quality of life, PLHIV are at increased risk for suicidal ideation and behavior.1 In addition, some ARTs have been thought to exacerbate or increase the risk for depression and suicidality.[10,11] Thus, need exists for the early identification of suicidal ideation/ behavior and intervention, as recommended by FDA guidelines.[4] Repeat assessment can help identify the interplay between baseline risk, disease risk, and possible drug effect on such risk. An evaluation of suicidal ideation/behavior is monitored in ViiV Healthcare interventional studies via the eC-SSRS. The eC-SSRS uses a direct, objective, validated data collection method. Upon initial inclusion into ViiV Healthcare clinical studies, some site staff felt the eC-SSRS was time consuming, upsetting to patients, redundant, or could interfere with their own clinical safety monitoring of study participants. It is important for individuals administering a suicidality assessment to have confidence in the utility of the instrument. ViiV Healthcare created the VEQ to understand site perceptions on administering the eC-SSRS and the value of the eC-SSRS for monitoring suicidal risk. Risk is the possibility that something bad or unpleasant (e.g., injury, loss) will happen in the future, whereas the eC-SSRS assesses current or retrospective suicidality. Indeed, acknowledgment of a patient’s recent or current suicidal ideation or intent is indicative of a risk for this behavior after the clinic visit, unless addressed by medical personnel. At the baseline visit, any ideation within the prior two months was considered “recent” enough to trigger an alert to the site because this could indicate risk that the patient could have ongoing issues or that these issues might recur. To our knowledge, the VEQ is the first survey of its type to obtain results from clinical study site personnel on their perspectives on implementation of the eC-SSRS.

Our results showed that most VEQ respondents were first-time administrators of the eC-SSRS. Most responded that the eC-SSRS was easy to administer, and most of those with prior eC-SSRS experience said that its administration became easier over time. Use of the phone version is important to maintain, as approximately one-third of respondents said their patients would not be successful in completing a web-based survey. Access to Wi-Fi and literacy/comprehension difficulties could be among the reasons why some sites were unable to complete the eC-SSRS via the web. Administration of the eC-SSRS via the phone versus the internet/web did not appear to impact the VEQ results. Most respondents said the eC-SSRS agreed with their clinical assessment. One-half of respondents reported that the eC-SSRS accurately assessed the risk for suicidality, 14 percent said the survey identified a previously unreported risk for suicidality, and one-quarter felt it overestimated the risk. It is possible that sites reporting that the eC-SSRS overestimated the risk for suicidality were sites with false/unconfirmed alerts. Additional study participant training and provision of instruction/reminder cards prior to each assessment might reduce the incidence of false positives in future studies. Few VEQ respondents reported that their patients refused to take the eC-SSRS; however, this low rate of refusal could be because successful completion of the eC-SSRS was a requirement for study entry and study continuation. The rate of false-positive alerts was higher post-baseline compared to baseline and could have been caused by confusion in recording lifetime history events as occurring since the last call. The eC-SSRS results from ARIA and STRIIVING were consistent with the clinical assessment data, with a pooled baseline prevalence of suicidality at nine percent. Overall, respondents felt confident their study patients were being assessed accurately for suicidal risk and that the eC-SSRS was helping them to manage this risk appropriately. Implementation of the eC-SSRS provides data for risk analysis, as help in identifying any recent or new risk of suicidal ideation/ behavior, which can prompt further clinical assessment.


All listed authors meet the criteria for authorship set forth by the International Committee for Medical Journal Editors. Editorial assistance was provided under the direction of the authors by Sherri Damlo, MedThink SciCom and was funded by ViiV Healthcare.


  1. Weber R, Ruppik M, Rickenbach M, et al. Decreasing mortality and changing patterns of causes of death in the Swiss HIV Cohort Study. HIV Med. 2013;14:195–207.
  2. Jia CX, Mehlum L, Qin P. AIDS/HIV infection, comorbid psychiatric illness, and risk for subsequent suicide: a nationwide register linkage study. J Clin Psychiatry. 2012;73:1315– 1321.
  3. Catalan J, Harding R, Sibley E, et al. HIV infection and mental health: suicidal behaviour- -systematic review. Psychol Health Med. 2011;16:588–611.
  4. Guidance for industry: suicidal ideation and behavior: prospective assessment of occurrence in clinical trials [draft guidance]. US Food and Drug Administration website. August 2012. Accessed 30 Oct 2017.
  5. Posner K, Brent D, Lucas C, et al. Columbia- Suicide Severity Rating Scale (C-SSRS). Columbia University website. June 23, 2010. Accessed 30 Oct 2017.
  6. Giddens JM, Sheehan KH, Sheehan DV. The Columbia-Suicide Severity Rating Scale (C-SSRS): has the “gold standard” become a liability? Innov Clin Neurosci. 2014;11:66–80.
  7. Mundt JC, Greist JH, Gelenberg AJ, et al. Feasibility and validation of a computer-automated Columbia-Suicide Severity Rating Scale using interactive voice response technology. J Psychiatr Res. 2010;44:1224–1228.
  8. Orrell C, Hagins DP, Belonosova E, et al; ARIA study team. Fixed-dose combination dolutegravir, abacavir, and lamivudine versus ritonavir-boosted atazanavir plus tenofovir disoproxil fumarate and emtricitabine in previously untreated women with HIV-1 infection (ARIA): week 48 results from a randomised, open-label, non-inferiority, phase 3b study. [Erratum appears in Lancet HIV. 2017;4:e546]. Lancet HIV. 2017;4:e536–e546.
  9. Trottier B, Lake JE, Logue K, et al. Dolutegravir/ abacavir/lamivudine versus current ART in virally suppressed patients (STRIIVING): a 48-week, randomized, non-inferiority, open-label, phase IIIb study. Antivir Ther. 2017;22:295–305.
  10. Mollan KR, Smurzynski M, Eron JJ, et al. Association between efavirenz as initial therapy for HIV-1 infection and increased risk for suicidal ideation or attempted or completed suicide: an analysis of trial data. Ann Intern Med. 2014;161:1–10.
  11. Treisman GJ, Kaplin AI. Neurologic and psychiatric complications of antiretroviral agents. AIDS. 2002;16:1201–1215.

APPENDIX 1. The ViiV eC-SSRS Perspectives Questionnaire (VEQ)


Site Perspectives on Implementation of the electronic Columbia-Suicide Severity Rating scale (eC-SSRS)

Are you the (check one)

  • Investigator
  • Study coordinator
  • Other (Specify)

Which version of the electronic Columbia-Suicide Severity Rating Scale (eC-SSRS) survey did you use? Check one:

  • Phone
  • Web
  • Both

If the eC-SSRS phone survey was not available, would all your subjects be successful in completing a webbased survey?

  • YES
  • NO

Is this the first time administering a suicidality assessment survey?

  • YES
  • NO

-If NO, has it become easier for you to implement the eC-SSRS survey over time?

  • YES
  • NO

Overall, on a scale of 1 to 5, how easy was it to administer the eC-SSRS survey? (Was it easy to explain the reason for the survey, was it easy to explain how to take the survey? Consider any other support/help you had to provide for completion of the survey)

  • 1-Very easy
  • 2-Easy
  • 3-Medium
  • 4-Hard
  • 5-Very hard

On a scale of 1 to 5, how easy was it to register the subject in the system?

  • 1-Very easy
  • 2-Easy
  • 3-Medium
  • 4-Hard
  • 5-Very hard

For what proportion of subjects at your site did the eC-SSRS survey result agree with your clinical assessment of the subject?

  • <25%
  • 25–<50%
  • 50–<75%
  • 75–<90%
  • >90%

In your opinion does the eC-SSRS survey (check all that apply):

  • Overestimate the risk for suicidality compared to clinical assessment
  • Underestimate the risk for suicidality compared to clinical assessment
  • Accurately assess the risk compared to clinical assessment
  • Identify previously unreported risk for suicidality

Of the total number of subjects successfully screened and deemed eligible to participate at your site, approximately what proportion refused to complete the eC-SSRS survey?

  • <25%
  • 25–<50%
  • 50–<75%
  • 75–<90%
  • >90%

If you had subjects that refused to complete the eC-SSRS survey, what were their reported reasons for refusal? (check all that apply)

  • Privacy/confidentiality issues
  • Too time consuming
  • Other
  • Not applicable

On a scale of 1 to 5, how confident are you that your subjects are being assessed accurately for suicidal risk and that the eC-SSRS survey is helping you manage this risk appropriately?

  • 1-Very confident
  • 2-Somewhat confident
  • 3-Neutral
  • 4-Minimal confidence
  • 5-Not at all confident

Tags: , , , ,

Category: Current Issue, Original Research, Suicidality

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.