# CounselBench: A Large-Scale Expert Evaluation and Adversarial Benchmarking of Large Language Models in Mental Health Question Answering

arXiv: https://arxiv.org/abs/2506.08584 · HTML: https://mlsanity.com/paper/2506.08584/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 91 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Builds on (most cited first, 30 of 32)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs](https://mlsanity.com/paper/2404.16873/) (2024)
- [Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks](https://mlsanity.com/paper/2410.20266/) (2024)
- [PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Support](https://mlsanity.com/paper/2106.01702/) (2021)
- [Dreaddit: A Reddit Dataset for Stress Analysis in Social Media](https://mlsanity.com/paper/1911.00133/) (2019)
- [Dialogue Response Ranking Training with Large-Scale Human Feedback Data](https://mlsanity.com/paper/2009.06978/) (2020)
- [A Computational Framework for Behavioral Assessment of LLM Therapists](https://mlsanity.com/paper/2401.00820/) (2024)
- [MedExQA: Medical Question Answering Benchmark with Multiple Explanations](https://mlsanity.com/paper/2406.06331/) (2024)
- [The opportunities and risks of large language models in mental health](https://mlsanity.com/paper/2403.14814/) (2024)
- [CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering](https://mlsanity.com/paper/2403.16008/) (2024)
- [Improving Reinforcement Learning from Human Feedback with Efficient Reward Model Ensemble](https://mlsanity.com/paper/2401.16635/) (2024)
- [On the State of Social Media Data for Mental Health Research](https://mlsanity.com/paper/2011.05233/) (2020)
- [MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming](https://mlsanity.com/paper/2505.17147/) (2025)
- [Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation](https://mlsanity.com/paper/2406.11852/) (2024)
- [A Benchmark for Long-Form Medical Question Answering](https://mlsanity.com/paper/2411.09834/) (2024)
- [The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support](https://mlsanity.com/paper/2401.14362/) (2024)
- [Towards a Client-Centered Assessment of LLM Therapists by Client Simulation](https://mlsanity.com/paper/2406.12266/) (2024)
- [Can AI Relate: Testing Large Language Model Response for Mental Health Support](https://mlsanity.com/paper/2405.12021/) (2024)
- [Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs](https://mlsanity.com/paper/2402.11958/) (2024)
- [The Role of AI in Peer Support for Young People: A Study of Preferences for Human- and AI-Generated Responses](https://mlsanity.com/paper/2405.02711/) (2024)
- [Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations](https://mlsanity.com/paper/2504.12691/) (2025)
- [De-identification of clinical free text using natural language processing: A systematic review of current approaches](https://mlsanity.com/paper/2312.03736/) (2023)
- [MHQA: A Diverse, Knowledge Intensive Mental Health Question Answering Challenge for Language Models](https://mlsanity.com/paper/2502.15418/) (2025)
- [Cooperation in the Gig Economy: Insights from Upwork Freelancers](https://mlsanity.com/paper/2301.08808/) (2023)
- [Evaluation of Large Language Models for Summarization Tasks in the Medical Domain: A Narrative Review](https://mlsanity.com/paper/2409.18170/) (2024)

## Built on by (most cited first, 21 of 21)

- [AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?](https://mlsanity.com/paper/2603.19574/) (2026)
- [MiraMind: Benchmarking Reliable Mental Health Reasoning beyond Answer Accuracy](https://mlsanity.com/paper/2512.09636/) (2025)
- [T-FIX: Text-Based Explanations with Features Interpretable to eXperts](https://mlsanity.com/paper/2511.04070/) (2025)
- [Expert Evaluation and the Limits of Human Feedback in Mental Health AI Safety Testing](https://mlsanity.com/paper/2601.18061/) (2026)
- [LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback](https://mlsanity.com/paper/2605.30273/) (2026)
- [Auditing Stealth Sycophancy in Mental-Health Dialogue: Structured Clinical-State Diagnostics and Clean Matched Benchmarks](https://mlsanity.com/paper/2605.03472/) (2026)
- [Mental Health AI Safety Claims Must Preserve Temporal Evidence](https://mlsanity.com/paper/2605.08827/) (2026)
- [SafeTalkCoach: Diversity-Driven Multi-Agent Simulation for Parent-Teen Health Conversations](https://mlsanity.com/paper/2602.00017/) (2026)
- [TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation](https://mlsanity.com/paper/2602.22775/) (2026)
- [CounselReflect: Opportunities and Challenges for Designing Tools to Support Self-Reflection on Mental Health and Well-Being Conversations with AI](https://mlsanity.com/paper/2603.29429/) (2026)
- [MHGraphBench: Knowledge Graph-Grounded Benchmarking of Mental Health Knowledge in Large Language Models](https://mlsanity.com/paper/2605.15589/) (2026)
- [AI Content Moderation in Therapy Conversations](https://mlsanity.com/paper/2605.25454/) (2026)
- [Accountable Human-AI Deliberation with LLMs: Scaling Collective Intelligence through Symbiotic Scaffolding](https://mlsanity.com/paper/2605.26940/) (2026)
- [Does Capability Transfer to Subjective Behavior -- and Would Our Instruments Tell Us? A Self-Evolving, Trust-by-Construction Evaluation Paradigm](https://mlsanity.com/paper/2605.27914/) (2026)
- [Psi-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues](https://mlsanity.com/paper/2606.02754/) (2026)
- [Toward Contemplative LLM: A Modular Framework for Evaluating and Enhancing LLM Alignment in Mental Health](https://mlsanity.com/paper/2607.10871/) (2026)
- [CARE-MH: Towards Unified, Reproducible, and Comparable Evaluation of Mental Health LLMs](https://mlsanity.com/paper/2607.24754/) (2026)
- [Detection != Reliable Control: Decodable Empathy Directions Yield at Most Partial Shifts in Automated Empathy Scores](https://mlsanity.com/paper/2608.24901/) (2026)
- [PeerPen: AI-Assisted Writing for Online Mental Health Peer Support](https://mlsanity.com/paper/2609.14886/) (2026)
- [K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations](https://mlsanity.com/paper/2609.15855/) (2026)
- [Empathy Is Not the Scapegoat for Declines in Factuality and Reliability](https://mlsanity.com/paper/2610.18541/) (2026)

## Abstract (arXiv)

Medical question answering (QA) benchmarks often focus on multiple-choice or fact-based tasks, leaving open-ended answers to real patient questions underexplored. This gap is particularly critical in mental health, where patient questions often mix symptoms, treatment concerns, and emotional needs, requiring answers that balance clinical caution with contextual sensitivity. We present CounselBench, a large-scale benchmark developed with 100 mental health professionals to evaluate and stress-test large language models (LLMs) in realistic help-seeking scenarios. The first component, CounselBench-EVAL, contains 2,000 expert evaluations of answers from GPT-4, LLaMA 3, Gemini, and online human therapists on patient questions from the public forum CounselChat. Each answer is rated across six clinically grounded dimensions, with span-level annotations and written rationales. Expert evaluations show that while LLMs achieve high scores on several dimensions, they also exhibit recurring issues, including unconstructive feedback, overgeneralization, and limited personalization or relevance. Responses were frequently flagged for safety risks, most notably unauthorized medical advice. Follow-up experiments show that LLM judges systematically overrate model responses and overlook safety concerns identified by human experts. To probe failure modes more directly, we construct CounselBench-Adv, an adversarial dataset of 120 expert-authored mental health questions designed to trigger specific model issues. Expert evaluation of 1,080 responses from nine LLMs reveals consistent, model-specific failure patterns. Together, CounselBench establishes a clinically grounded framework for benchmarking LLMs in mental health QA.