Where do LLMs Fall Short in CBT-Guided Affective Reasoning?
On the map: LLM reasoning → Clinical and biomedical applications
Abstract (arXiv)
Cognitive Behavioral Therapy (CBT) provides a structured framework for understanding a user's mental state by examining the interaction between cognitive and behavioral factors. However, out-of-the-box LLMs respond fluently and empathetically, yet collapse into validation & reflection, regardless of what the user actually needs. They know theoretical CBT (scoring up to 96% accuracy on licensing exam questions) but fail to apply it effectively. We explore this gap with a knowledge-guided framework that treats CBT dialogue as controlled affective reasoning: user narratives are decomposed into Beck's Cognitive Conceptualization structure, grounded in clinical SNOMED CT concepts validated via Natural Language Inference, and a Multiple Chain-of-Thought (MCoT) strategy selection between Validation & Reflection, Socratic Questioning, or Alternative Perspectives. To measure whether such guidance actually changes behavior, we introduce the Protocol Leverage Force (F), a behavior-level metric that captures how far an intervention shifts a model away from its default response. Across three open-weight LLMs and 14 RealCBT-derived case studies, evaluated with human experts, valence-arousal trajectories, and linguistic entrainment, F shows that simply introducing protocol definitions via single chain-of-thought prompting fails to change LLM behavior, while MCoT on these definitions guides strategy selection better. Still, the effect stays within 1% (approx. 1.2-1.3%), and all models remain biased toward Validation & Reflection. These results show CBT knowledge alone does not ensure effective application, giving the affective-computing community instrumentation to measure where LLMs fall short.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 5 of its 41 references that are in the map. First posted on arXiv 3 Jul 2026.
Builds on
5 of 41 references.
References · 5
- 2025Forecasting Rare Language Model Behaviorscited by 16
- 2024Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verificationcited by 13
- 2024CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapycited by 12
- 2025One Battle After Another: Probing LLMs' Limits on Multi-Turn Instruction Following with a Benchmark Evolving Frameworkcited by 9
- 2026Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretabilitycited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Assessing the Effectiveness of LLMs in Delivering Cognitive Behavioral Therapy
- 2024Dialectical Behavior Therapy Approach to LLM Prompting
- 2026CCD-CBT: Multi-Agent Therapeutic Interaction for CBT Guided by Cognitive Conceptualization Diagram
- 2023Chain of Empathy: Enhancing Empathetic Response of Large Language Models Based on Psychotherapy Models
- 2025Does "Reasoning" with Large Language Models Improve Recognizing, Generating, and Reframing Unhelpful Thoughts?
- 2026MACBT: A Multi-Agent Cognitive Behavioral Therapy Decision Support System with Longitudinal Memory
- 2026EFT-CoT: A Multi-Agent Chain-of-Thought Framework for Emotion-Focused Therapy
- 2025Multimodal Cognitive Reframing Therapy via Multi-hop Psychotherapeutic Reasoning