# A Benchmark for Understanding Dialogue Safety in Mental Health Support

arXiv: https://arxiv.org/abs/2307.16457 · HTML: https://mlsanity.com/paper/2307.16457/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 18 references that are in the map.
- First posted on arXiv 31 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://mlsanity.com/paper/2203.09509/) (2022)
- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack](https://mlsanity.com/paper/1908.06083/) (2019)
- [On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark](https://mlsanity.com/paper/2110.08466/) (2021)
- [Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts](https://mlsanity.com/paper/2108.11830/) (2021)
- [SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification](https://mlsanity.com/paper/2004.14454/) (2020)
- [EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training](https://mlsanity.com/paper/2203.09313/) (2022)
- [Ruddit: Norms of Offensiveness for English Reddit Comments](https://mlsanity.com/paper/2106.05664/) (2021)
- [Micromodels for Efficient, Explainable, and Reusable Systems: A Case Study on Mental Health](https://mlsanity.com/paper/2109.13770/) (2021)
- [Understanding Client Reactions in Online Mental Health Counseling](https://mlsanity.com/paper/2306.15334/) (2023)
- [Robots-Dont-Cry: Understanding Falsely Anthropomorphic Utterances in Dialog Systems](https://mlsanity.com/paper/2210.12429/) (2022)

## Built on by (most cited first, 13 of 13)

- [ChatCounselor: A Large Language Models for Mental Health Support](https://mlsanity.com/paper/2309.15461/) (2023)
- [Large Language Models in Mental Health Care: a Scoping Review](https://mlsanity.com/paper/2401.02984/) (2024)
- [PsycoLLM: Enhancing LLM for Psychological Understanding and Evaluation](https://mlsanity.com/paper/2407.05721/) (2024)
- [Rethinking Large Language Models in Mental Health Applications](https://mlsanity.com/paper/2311.11267/) (2023)
- [PsyEval: A Suite of Mental Health Related Tasks for Evaluating Large Language Models](https://mlsanity.com/paper/2311.09189/) (2023)
- [A Comprehensive Evaluation of Large Language Models on Mental Illnesses](https://mlsanity.com/paper/2409.15687/) (2024)
- [Labeling Free-text Data using Language Model Ensembles](https://mlsanity.com/paper/2501.08413/) (2025)
- [A Risk Ontology for Evaluating AI-Powered Psychotherapy Virtual Agents](https://mlsanity.com/paper/2505.15108/) (2025)
- [Understanding Risk and Dependency in AI Chatbot Use from User Discourse](https://mlsanity.com/paper/2602.09339/) (2026)
- [TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health](https://mlsanity.com/paper/2603.03047/) (2026)
- [AERIC: Anticipatory Hidden-State Monitoring for Implicit Harmful Dialogue](https://mlsanity.com/paper/2605.23974/) (2026)
- [K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations](https://mlsanity.com/paper/2609.15855/) (2026)
- [From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Health](https://mlsanity.com/paper/2609.25186/) (2026)

## Abstract (arXiv)

Dialogue safety remains a pervasive challenge in open-domain human-machine interaction. Existing approaches propose distinctive dialogue safety taxonomies and datasets for detecting explicitly harmful responses. However, these taxonomies may not be suitable for analyzing response safety in mental health support. In real-world interactions, a model response deemed acceptable in casual conversations might have a negligible positive impact on users seeking mental health support. To address these limitations, this paper aims to develop a theoretically and factually grounded taxonomy that prioritizes the positive impact on help-seekers. Additionally, we create a benchmark corpus with fine-grained labels for each dialogue session to facilitate further research. We analyze the dataset using popular language models, including BERT-base, RoBERTa-large, and ChatGPT, to detect and understand unsafe responses within the context of mental health support. Our study reveals that ChatGPT struggles to detect safety categories with detailed safety definitions in a zero- and few-shot paradigm, whereas the fine-tuned model proves to be more suitable. The developed dataset and findings serve as valuable benchmarks for advancing research on dialogue safety in mental health support, with significant implications for improving the design and deployment of conversation agents in real-world applications. We release our code and data here: https://github.com/qiuhuachuan/DialogueSafety.