TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health
Not on the map. Why?
Abstract (arXiv)
While Large Language Models (LLMs) demonstrate significant potential in providing accessible mental health support, their practical deployment raises critical trustworthiness concerns due to the domains high-stakes and safety-sensitive nature. Existing evaluation paradigms for general-purpose LLMs fail to capture mental health-specific requirements, highlighting an urgent need to prioritize and enhance their trustworthiness. To address this, we propose TrustMH-Bench, a holistic framework designed to systematically quantify the trustworthiness of mental health LLMs. By establishing a deep mapping from domain-specific norms to quantitative evaluation metrics, TrustMH-Bench evaluates models across eight core pillars: Reliability, Crisis Identification and Escalation, Safety, Fairness, Privacy, Robustness, Anti-sycophancy, and Ethics. We conduct extensive experiments across six general-purpose LLMs and six specialized mental health models. Experimental results indicate that the evaluated models underperform across various trustworthiness dimensions in mental health scenarios, revealing significant deficiencies. Notably, even generally powerful models (e.g., GPT-5.1) fail to maintain consistently high performance across all dimensions. Consequently, systematically improving the trustworthiness of LLMs has become a critical task. Our data and code are released.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 73 references that are in the map. First posted on arXiv 3 Mar 2026.
Builds on
34 of 73 references.
References · 34
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
- 2023A Survey on Evaluation of Large Language Modelscited by 1,227
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
- 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
- 2021Towards Emotional Support Dialog Systemscited by 294
- 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
- 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
- 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 159
- 2021PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Supportcited by 92
22 more
- 2024CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counselingcited by 88
- 2019Dreaddit: A Reddit Dataset for Stress Analysis in Social Mediacited by 81
- 2018SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditionscited by 73
- 2024The opportunities and risks of large language models in mental healthcited by 54
- 2022D4: a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chatcited by 37
- 2024PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counselingcited by 37
- 2022CAMS: An Annotated Corpus for Causal Analysis of Mental Health Issues in Social Media Postscited by 34
- 2020Suicidal Ideation and Mental Disorder Detection with Attentive Relation Networkscited by 28
- 2023Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBenchcited by 28
- 2025When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluationcited by 19
- 2023An Annotated Dataset for Explainable Interpersonal Risk Factors of Mental Disturbance in Social Media Postscited by 16
- 2023A Benchmark for Understanding Dialogue Safety in Mental Health Supportcited by 13
- 2023PsyEval: A Suite of Mental Health Related Tasks for Evaluating Large Language Modelscited by 13
- 2024CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapycited by 12
- 2024MentalGLM Series: Explainable Large Language Models for Mental Health Analysis on Chinese Social Mediacited by 9
- 2025Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMscited by 6
- 2025CARE-Bench: A Benchmark of Diverse Client Simulations Guided by Expert Principles for Evaluating LLMs in Psychological Counselingcited by 6
- 2024Unveiling and Mitigating Bias in Mental Health Analysis with Large Language Modelscited by 5
- 2024A Framework for Evaluating Appropriateness, Trustworthiness, and Safety in Mental Wellness AI Chatbotscited by 5
- 2025PsychBench: A comprehensive and professional benchmark for evaluating the performance of LLM-assisted psychiatric clinical practicecited by 5
- 2025DecoupledESC: Enhancing Emotional Support Generation via Strategy-Response Decoupled Preference Optimizationcited by 5
- 2025Sycophantic AI Decreases Prosocial Intentions and Promotes Dependencecited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.