log in  |  register  |  feedback?  |  help  |  web accessibility
Natural Language Psychometrics: Measuring Suicide Risk Factors in Text
Wednesday, March 29, 2023, 11:00 am-12:00 pm Calendar
  • You are subscribed to this talk through .
  • You are watching this talk through .
  • You are subscribed to this talk. (unsubscribe, watch)
  • You are watching this talk. (unwatch, subscribe)
  • You are not subscribed to this talk. (watch, subscribe)

Mental health rating scales are used to diagnose and assess risk. They are often retrospective (e.g., ask about symptoms in the past), rely on an individual accurately interpreting the wording of items, and can miss key risk factors if not included in the questionnaire. When humans describe their symptoms using their own natural language, they can potentially describe many more risk factors in an ecological way. The challenge is using psychometric properties to reliably measure symptoms from text as has been done with rating scales.
I will present a series of methods for measuring psychological constructs from text including semi-automated ways of generating a lexicon for a given construct as well as few-shot learning and semantic-similarity approaches that use embeddings to overcome lexicons' need for an exact match. I report how well these methods measure 38 known risk factors for suicidal thoughts and behaviors, which highlight the need for explainable and fair assessments. I validate these measurements of risk factors by using them to predict the severity of a crisis counseling session using a dataset of over 100k sessions. I also test how well each method is able to quantify one of 12 types of crisis (e.g., suicide, sexual assault, anxiety, eating disorders) as annotated by counselors.
In a second study, I apply these methods to longitudinal data, an ecological momentary assessment study (N=106), and assess their reliability. Suicidal individuals responded to rating scales about their mental health and also described the specific content of their suicidal ideation daily for 22 days on average. Our results allow us to provide an answer to a fundamental question: what do people actually think about when they think about suicide? And we demonstrate how capturing the dynamics of specific thoughts over time can help assess risk.


Daniel Low is a PhD Candidate in the Speech and Hearing Bioscience and Technology program at Harvard University. He is part of the Senseable Intelligence Group, a biomarker machine learning lab at MIT, and the Nock Lab, a suicide research group at Harvard University. He is working on detecting mental health symptoms from text and speech data using natural language processing, speech signal processing, machine learning, and causal inference. His focus is on suicidal thoughts and behaviors as well as states of well-being and insight caused by meditation and psilocybin. He uses data from ecological momentary assessments of hospitalized and nonhospitalized individuals, social media, and clinical trials. His overarching goal is to develop technology that can provide individuals and clinicians with digital assessments and interventions to overcome barriers to treatment such as cost and sociocultural inequity. His work is funded by a RallyPoint Fellowship, the NIH Common Fund Bridge2AI program, and an Amelia Peabody Professional Development Award.

This talk is organized by Rachel Rudinger