# CARE-Bench: A Benchmark of Diverse Client Simulations Guided by Expert Principles for Evaluating LLMs in Psychological Counseling

arXiv: https://arxiv.org/abs/2511.09407 · HTML: https://mlsanity.com/paper/2511.09407/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2025.

## Built on by (most cited first, 6 of 6)

- [Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling](https://mlsanity.com/paper/2604.04842/) (2026)
- [MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models](https://mlsanity.com/paper/2604.17730/) (2026)
- [PsychePass: Calibrating LLM Therapeutic Competence via Trajectory-Anchored Tournaments](https://mlsanity.com/paper/2601.20330/) (2026)
- [TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health](https://mlsanity.com/paper/2603.03047/) (2026)
- [PSI-Bench: Interpretable and Clinically Meaningful Evaluation of Depression Patient Simulators](https://mlsanity.com/paper/2604.25840/) (2026)
- [ENPMR-Bench: Benchmarking Proactive Memory Retrieval for Emotional Support Agents](https://mlsanity.com/paper/2605.27240/) (2026)

## Abstract (arXiv)

The mismatch between the growing demand for psychological counseling and the limited availability of services has motivated research into the application of Large Language Models (LLMs) in this domain. Consequently, there is a need for a robust and unified benchmark to assess the counseling competence of various LLMs. Existing works, however, are limited by unprofessional client simulation, static question-and-answer evaluation formats, and unidimensional metrics. These limitations hinder their effectiveness in assessing a model's comprehensive ability to handle diverse and complex clients. To address this gap, we introduce CARE-Bench, a dynamic and interactive automated benchmark. It is built upon diverse client profiles derived from real-world counseling cases and simulated according to expert guidelines. CARE-Bench provides a multidimensional performance evaluation grounded in established psychological scales. Using CARE-Bench, we evaluate several general-purpose LLMs and specialized counseling models, revealing their current limitations. In collaboration with psychologists, we conduct a detailed analysis of the reasons for LLMs' failures when interacting with clients of different types, which provides directions for developing more comprehensive, universal, and effective counseling models.