# TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health

arXiv: https://arxiv.org/abs/2603.03047 · HTML: https://mlsanity.com/paper/2603.03047/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 73 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Builds on (most cited first, 30 of 34)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Towards Emotional Support Dialog Systems](https://mlsanity.com/paper/2106.01144/) (2021)
- [TrustLLM: Trustworthiness in Large Language Models](https://mlsanity.com/paper/2401.05561/) (2024)
- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Support](https://mlsanity.com/paper/2106.01702/) (2021)
- [CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling](https://mlsanity.com/paper/2405.16433/) (2024)
- [Dreaddit: A Reddit Dataset for Stress Analysis in Social Media](https://mlsanity.com/paper/1911.00133/) (2019)
- [SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions](https://mlsanity.com/paper/1806.05258/) (2018)
- [The opportunities and risks of large language models in mental health](https://mlsanity.com/paper/2403.14814/) (2024)
- [PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counseling](https://mlsanity.com/paper/2412.13660/) (2024)
- [D4: a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chat](https://mlsanity.com/paper/2205.11764/) (2022)
- [CAMS: An Annotated Corpus for Causal Analysis of Mental Health Issues in Social Media Posts](https://mlsanity.com/paper/2207.04674/) (2022)
- [Suicidal Ideation and Mental Disorder Detection with Attentive Relation Networks](https://mlsanity.com/paper/2004.07601/) (2020)
- [Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench](https://mlsanity.com/paper/2310.01386/) (2023)
- [When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation](https://mlsanity.com/paper/2510.19032/) (2025)
- [An Annotated Dataset for Explainable Interpersonal Risk Factors of Mental Disturbance in Social Media Posts](https://mlsanity.com/paper/2305.18727/) (2023)
- [A Benchmark for Understanding Dialogue Safety in Mental Health Support](https://mlsanity.com/paper/2307.16457/) (2023)
- [PsyEval: A Suite of Mental Health Related Tasks for Evaluating Large Language Models](https://mlsanity.com/paper/2311.09189/) (2023)
- [CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy](https://mlsanity.com/paper/2410.13218/) (2024)
- [MentalGLM Series: Explainable Large Language Models for Mental Health Analysis on Chinese Social Media](https://mlsanity.com/paper/2410.10323/) (2024)
- [Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs](https://mlsanity.com/paper/2509.24857/) (2025)
- [Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence](https://mlsanity.com/paper/2510.01395/) (2025)
- [CARE-Bench: A Benchmark of Diverse Client Simulations Guided by Expert Principles for Evaluating LLMs in Psychological Counseling](https://mlsanity.com/paper/2511.09407/) (2025)

## Built on by (most cited first, 2 of 2)

- [MHGraphBench: Knowledge Graph-Grounded Benchmarking of Mental Health Knowledge in Large Language Models](https://mlsanity.com/paper/2605.15589/) (2026)
- [DistressBench: Evaluating Multi-Dimensional Crisis Support in Large Language Models](https://mlsanity.com/paper/2610.12758/) (2026)

## Abstract (arXiv)

While Large Language Models (LLMs) demonstrate significant potential in providing accessible mental health support, their practical deployment raises critical trustworthiness concerns due to the domains high-stakes and safety-sensitive nature. Existing evaluation paradigms for general-purpose LLMs fail to capture mental health-specific requirements, highlighting an urgent need to prioritize and enhance their trustworthiness. To address this, we propose TrustMH-Bench, a holistic framework designed to systematically quantify the trustworthiness of mental health LLMs. By establishing a deep mapping from domain-specific norms to quantitative evaluation metrics, TrustMH-Bench evaluates models across eight core pillars: Reliability, Crisis Identification and Escalation, Safety, Fairness, Privacy, Robustness, Anti-sycophancy, and Ethics. We conduct extensive experiments across six general-purpose LLMs and six specialized mental health models. Experimental results indicate that the evaluated models underperform across various trustworthiness dimensions in mental health scenarios, revealing significant deficiencies. Notably, even generally powerful models (e.g., GPT-5.1) fail to maintain consistently high performance across all dimensions. Consequently, systematically improving the trustworthiness of LLMs has become a critical task. Our data and code are released.