# CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy

arXiv: https://arxiv.org/abs/2410.13218 · HTML: https://mlsanity.com/paper/2410.13218/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 9 of 9)

- [Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction](https://mlsanity.com/paper/2305.02466/) (2023)
- [Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach](https://mlsanity.com/paper/2101.07714/) (2021)
- [Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles](https://mlsanity.com/paper/2407.00870/) (2024)
- [Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring](https://mlsanity.com/paper/2310.15461/) (2023)
- [Training Models to Generate, Recognize, and Reframe Unhelpful Thoughts](https://mlsanity.com/paper/2307.02768/) (2023)
- [Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting](https://mlsanity.com/paper/2310.07146/) (2023)
- [Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation](https://mlsanity.com/paper/2406.11852/) (2024)
- [Rethinking Large Language Models in Mental Health Applications](https://mlsanity.com/paper/2311.11267/) (2023)
- [mhGPT: A Lightweight Generative Pre-Trained Transformer for Mental Health Text Analysis](https://mlsanity.com/paper/2408.08261/) (2024)

## Built on by (most cited first, 12 of 12)

- [Preference Learning Unlocks LLMs' Psycho-Counseling Skills](https://mlsanity.com/paper/2502.19731/) (2025)
- [MindEval: Benchmarking Language Models on Multi-turn Mental Health Support](https://mlsanity.com/paper/2511.18491/) (2025)
- [PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor](https://mlsanity.com/paper/2604.00931/) (2026)
- [TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health](https://mlsanity.com/paper/2603.03047/) (2026)
- [TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots](https://mlsanity.com/paper/2603.18008/) (2026)
- [Auditing Stealth Sycophancy in Mental-Health Dialogue: Structured Clinical-State Diagnostics and Clean Matched Benchmarks](https://mlsanity.com/paper/2605.03472/) (2026)
- [Where do LLMs Fall Short in CBT-Guided Affective Reasoning?](https://mlsanity.com/paper/2607.02885/) (2026)
- [MentalHospital: A Virtual Environment for Evaluating Psychiatric Clinical Encounters](https://mlsanity.com/paper/2607.08257/) (2026)
- [When LLM Therapists Become Salespeople: Evaluating Large Language Models for Ethical Motivational Interviewing](https://mlsanity.com/paper/2503.23566/) (2025)
- [CBT-Audio: Evaluating Audio Language Models for Patient-Side Distress Intensity Estimation in CBT Session Recordings](https://mlsanity.com/paper/2605.17370/) (2026)
- [Towards Understanding and Measuring COGNITIVE ATROPHY in LLM Behaviour](https://mlsanity.com/paper/2606.18129/) (2026)
- [From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Health](https://mlsanity.com/paper/2609.25186/) (2026)

## Abstract (arXiv)

There is a significant gap between patient needs and available mental health support today. In this paper, we aim to thoroughly examine the potential of using Large Language Models (LLMs) to assist professional psychotherapy. To this end, we propose a new benchmark, CBT-BENCH, for the systematic evaluation of cognitive behavioral therapy (CBT) assistance. We include three levels of tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of multiple-choice questions; II: Cognitive model understanding, with the tasks of cognitive distortion classification, primary core belief classification, and fine-grained core belief classification; III: Therapeutic response generation, with the task of generating responses to patient speech in CBT therapy sessions. These tasks encompass key aspects of CBT that could potentially be enhanced through AI assistance, while also outlining a hierarchy of capability requirements, ranging from basic knowledge recitation to engaging in real therapeutic conversations. We evaluated representative LLMs on our benchmark. Experimental results indicate that while LLMs perform well in reciting CBT knowledge, they fall short in complex real-world scenarios requiring deep analysis of patients' cognitive structures and generating effective responses, suggesting potential future work.