# CRADLE Bench: A Clinician-Annotated Benchmark for Multi-Faceted Mental Health Crisis and Safety Risk Detection

arXiv: https://arxiv.org/abs/2510.23845 · HTML: https://mlsanity.com/paper/2510.23845/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 27 Oct 2025.

## Builds on (most cited first, 9 of 9)

- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation](https://mlsanity.com/paper/2406.11852/) (2024)
- [PsyGUARD: An Automated System for Suicide Detection and Risk Assessment in Psychological Counseling](https://mlsanity.com/paper/2409.20243/) (2024)
- [Leveraging Large Language Models for Suicide Detection on Social Media with Limited Labels](https://mlsanity.com/paper/2410.04501/) (2024)
- [Just a Scratch: Enhancing LLM Capabilities for Self-harm Detection through Intent Differentiation and Emoji Interpretation](https://mlsanity.com/paper/2506.05073/) (2025)
- [Sensitive Content Classification in Social Media: A Holistic Resource and Evaluation](https://mlsanity.com/paper/2411.19832/) (2024)
- [SafeCity: Understanding Diverse Forms of Sexual Harassment Personal Stories](https://mlsanity.com/paper/1809.04739/) (2018)
- [TRUST: An LLM-Based Dialogue System for Trauma Understanding and Structured Assessments](https://mlsanity.com/paper/2504.21851/) (2025)
- [Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines](https://mlsanity.com/paper/2506.01329/) (2025)

## Built on by (most cited first, 8 of 8)

- ['Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions](https://mlsanity.com/paper/2606.04906/) (2026)
- [Expert Evaluation and the Limits of Human Feedback in Mental Health AI Safety Testing](https://mlsanity.com/paper/2601.18061/) (2026)
- [MentalSeek-Dx: Towards Progressive Hypothetico-Deductive Reasoning for Real-world Psychiatric Diagnosis](https://mlsanity.com/paper/2602.03340/) (2026)
- [Lost in Delusion: Examining LLM Safety Under User Delusions and Distress](https://mlsanity.com/paper/2606.00975/) (2026)
- [CounselReflect: Opportunities and Challenges for Designing Tools to Support Self-Reflection on Mental Health and Well-Being Conversations with AI](https://mlsanity.com/paper/2603.29429/) (2026)
- [Expert-Level Crisis Detection in Mental Health Conversations](https://mlsanity.com/paper/2606.10380/) (2026)
- [DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots](https://mlsanity.com/paper/2608.05004/) (2026)
- [K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations](https://mlsanity.com/paper/2609.15855/) (2026)

## Abstract (arXiv)

Detecting mental health crisis situations such as suicide ideation, rape, domestic violence, child abuse, and sexual harassment is a critical yet underexplored challenge for language models. When such situations arise during user--model interactions, models must reliably flag them, as failure to do so can have serious consequences. In this work, we introduce CRADLE BENCH, a benchmark for multi-faceted crisis detection. Unlike previous efforts that focus on a limited set of crisis types, our benchmark covers seven types defined in line with clinical standards and is the first to incorporate temporal labels. Our benchmark provides 600 clinician-annotated evaluation examples and 420 development examples, together with a training corpus of around 4K examples automatically labeled using a majority-vote ensemble of multiple language models, which significantly outperforms single-model annotation. We further fine-tune six crisis detection models on subsets defined by consensus and unanimous ensemble agreement, providing complementary models trained under different agreement criteria.