mlsanityData to 7 Oct 2026

Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluation

arXiv 2502.17899 ↗First posted 25 Feb 2025cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

We present a comprehensive evaluation framework for assessing Large Language Models' (LLMs) capabilities in suicide prevention, focusing on two critical aspects: the Identification of Implicit Suicidal ideation (IIS) and the Provision of Appropriate Supportive responses (PAS). We introduce ourdata, a novel dataset of 1,308 test cases built upon psychological frameworks including D/S-IAT and Negative Automatic Thinking, alongside real-world scenarios. Through extensive experiments with 8 widely used LLMs under different contextual settings, we find that current models struggle significantly with detecting implicit suicidal ideation and providing appropriate support, highlighting crucial limitations in applying LLMs to mental health contexts. Our findings underscore the need for more sophisticated approaches in developing and evaluating LLMs for sensitive psychological applications.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 20 of its 75 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

20 of 75 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  4. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  5. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  6. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  7. 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 143
  8. 2024Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversationcited by 55
  9. 2024The opportunities and risks of large language models in mental healthcited by 54
  10. 2021Deep Learning for Suicide and Depression Identification with Unsupervised Label Correctioncited by 45
  11. 2019Suicidal Ideation Detection: A Review of Machine Learning Methods and Applicationscited by 31
  12. 2024PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agentscited by 28
8 more
  1. 2024Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chatcited by 23
  2. 2023Rethinking Large Language Models in Mental Health Applicationscited by 16
  3. 2024Large Language Models for Wearable Sensor-Based Human Activity Recognition, Health Monitoring, and Behavioral Modeling: A Survey of Early Trends, Datasets, and Challengescited by 12
  4. 2024Leveraging Large Language Models for Suicide Detection on Social Media with Limited Labelscited by 9
  5. 2024Dialogue Language Model with Large-Scale Persona Data Engineeringcited by 9
  6. 2022ScAN: Suicide Attempt and Ideation Events Datasetcited by 8
  7. 2020Revealing semantic and emotional structure of suicide notes with cognitive network sciencecited by 6
  8. 2024Deep Learning and Large Language Models for Audio and Text Analysis in Predicting Suicidal Acts in Chinese Psychological Support Hotlinescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 11.

13 papers.

Sort:

By area:

  1. 2026AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitorcited by 8
  2. 2026AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluationcited by 4
  3. 2025The Compositional Architecture of Regret in Large Language Modelscited by 3
  4. 2026Responsible Evaluation of AI for Mental Healthcited by 2
  5. 2025Two-Stage Voting for Robust and Efficient Suicide Risk Detection on Social Mediacited by 0
  6. 2026Can Large Language Models Resolve Semantic Discrepancy in Self-Destructive Subcultures? Evidence from Jirai Keicited by 0
  7. 2026SuiChat-CN: Benchmarking Contextual Suicide Risk Assessment in Chinese Group Chatscited by 0
  8. 2026Multi-Adapter Representation Interventions via Energy Calibrationcited by 0
  9. 2026Expert-Level Crisis Detection in Mental Health Conversationscited by 0
  10. 2026Validation-Gated Causal Interventions for Interpreting High-Stakes Large Language Model Behavior: A Case Study in Suicidality Detectioncited by 0
  11. 2026Alignment Plausibility: A New Standard for Assuring AI in Healthcarecited by 0
  12. 2026Retrieval-Augmented Generation in LLMs for Mental Health: Quantifying the Incremental Contribution of Retrieval Within a Layered Safety Architecturecited by 0
1 more
  1. 2026From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Healthcited by 0