ChatThero: An LLM-Supported Chatbot for Behavior Change and Therapeutic Support in Addiction Recovery
Not on the map. Why?
Abstract (arXiv)
Substance use disorders (SUDs) affect millions of people, and relapses are common, requiring multi-session treatments. Access to care is limited, which contributes to the challenge of recovery support. We present ChatThero, an innovative low-cost, multi-session, stressor-aware, and memory-persistent autonomous language agent designed to facilitate long-term behavior change and therapeutic support in addiction recovery. Unlike existing work that mostly finetuned large language models (LLMs) on patient-therapist conversation data, ChatThero was trained in a multi-agent simulated environment that mirrors real therapy. We created anonymized patient profiles from recovery communities (e.g., Reddit). We classify patients as texttteasy, textttmedium, and textttdifficult, three scales representing their resistance to recovery. We created an external environment by introducing stressors (e.g., social determinants of health) to simulate real-world situations. We dynamically inject clinically-grounded therapeutic strategies (motivational interview and cognitive behavioral therapy). Our evaluation, conducted by both human (blinded clinicians) and LLM-as-Judge, shows that ChatThero is superior in empathy and clinical relevance. We show that stressor simulation improves robustness of ChatThero. Explicit stressors increase relapse-like setbacks, matching real-world patterns. We evaluate ChatThero with behavioral change metrics. On a 1--5 scale, ChatThero raises textttmotivation by +1.71 points (from 2.39 to 4.10) and textttconfidence by +1.67 points (from 1.52 to 3.19), substantially outperforming GPT-5. On textttdifficult patients, ChatThero reaches the success milestone with 26% fewer turns than GPT-5.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 34 of its 73 references that are in the map. First posted on arXiv 28 Aug 2025.
Builds on
34 of 73 references.
References · 34
- 2023GPT-4 Technical Reportcited by 18,396
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2024A Survey on LLM-as-a-Judgecited by 1,423
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
- 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
- 2023GPTScore: Evaluate as You Desirecited by 361
- 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
- 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 249
22 more
- 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 231
- 2024AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulatorcited by 101
- 2021PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Supportcited by 92
- 2024CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counselingcited by 89
- 2024Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principlescited by 73
- 2025Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Modelscited by 62
- 2024Cactus: Towards Psychological Counseling Conversations using Cognitive Behavioral Theorycited by 60
- 2024HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapycited by 47
- 2023SMILE: Single-turn to Multi-turn Inclusive Language Expansion via ChatGPT for Mental Health Supportcited by 46
- 2023A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluatorscited by 45
- 2024CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answeringcited by 45
- 2024Improving Medical Reasoning through Retrieval and Self-Reflection with Retrieval-Augmented Large Language Modelscited by 34
- 2024PsycoLLM: Enhancing LLM for Psychological Understanding and Evaluationcited by 26
- 2024RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Modelscited by 25
- 2024LLMs Can Simulate Standardized Patients via Agent Coevolutioncited by 22
- 2022StoryER: Automatic Story Evaluation via Ranking, Rating and Reasoningcited by 21
- 2023PaniniQA: Enhancing Patient Education Through Interactive Question Answeringcited by 21
- 2024Comparing Two Model Designs for Clinical Note Generation; Is an LLM a Useful Evaluator of Consistency?cited by 20
- 2024MedQA-CS: Objective Structured Clinical Examination (OSCE)-Style Benchmark for Evaluating LLM Clinical Skillscited by 17
- 2024Towards Social AI: A Survey on Understanding Social Interactionscited by 16
- 2024MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedbackcited by 14
- 2024Large Language Model-based Role-Playing for Personalized Medical Jargon Extractioncited by 4
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathycited by 0
- 2026Rethinking Patient Education as Multi-turn Multi-modal Interactioncited by 0
- 2026DischargeBench: Pre-Deployment Evaluation of LLM Discharge Educators through Persona-Grounded Patient Simulationcited by 0
- 2026Multi-Objective Aligned Small Language Model Framework for SUD Patient Dialogue Generationcited by 0
- 2026Multi-Objective Aligned Small Language Model Framework for SUD Patient Dialogue Generation6 Oct 2026
- 2026DischargeBench: Pre-Deployment Evaluation of LLM Discharge Educators through Persona-Grounded Patient Simulation22 Jul 2026
- 2026Rethinking Patient Education as Multi-turn Multi-modal Interaction16 Apr 2026
- 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathy12 Apr 2026