# Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs arXiv: https://arxiv.org/abs/2402.11958 ยท HTML: https://mlsanity.com/paper/2402.11958/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Builds on 12 of its 65 references that are in the map. - First posted on arXiv 19 Feb 2024. ## Builds on (most cited first, 12 of 12) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [Towards Emotional Support Dialog Systems](https://mlsanity.com/paper/2106.01144/) (2021) - [Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://mlsanity.com/paper/2305.15005/) (2023) - [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021) - [A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support](https://mlsanity.com/paper/2009.08441/) (2020) - [A Computational Framework for Behavioral Assessment of LLM Therapists](https://mlsanity.com/paper/2401.00820/) (2024) - [Evaluation of ChatGPT for NLP-based Mental Health Applications](https://mlsanity.com/paper/2303.15727/) (2023) - [Will Affective Computing Emerge from Foundation Models and General AI? A First Evaluation on ChatGPT](https://mlsanity.com/paper/2303.03186/) (2023) - [Observing Dialogue in Therapy: Categorizing and Forecasting Behavioral Codes](https://mlsanity.com/paper/1907.00326/) (2019) - [Rethinking Large Language Models in Mental Health Applications](https://mlsanity.com/paper/2311.11267/) (2023) - [Working Alliance Transformer for Psychotherapy Dialogue Classification](https://mlsanity.com/paper/2210.15603/) (2022) - [Towards Automated Real-time Evaluation in Text-based Counseling](https://mlsanity.com/paper/2203.03442/) (2022) ## Built on by (most cited first, 19 of 19) - [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024) - [Towards a Client-Centered Assessment of LLM Therapists by Client Simulation](https://mlsanity.com/paper/2406.12266/) (2024) - [CounselBench: A Large-Scale Expert Evaluation and Adversarial Benchmarking of Large Language Models in Mental Health Question Answering](https://mlsanity.com/paper/2506.08584/) (2025) - [When Large Language Models are Reliable for Judging Empathic Communication](https://mlsanity.com/paper/2506.10150/) (2025) - [Scaffolding Empathy: Training Counselors with Simulated Patients and Utterance-level Performance Visualizations](https://mlsanity.com/paper/2502.18673/) (2025) - [MIRROR: Multimodal Cognitive Reframing Therapy for Rolling with Resistance](https://mlsanity.com/paper/2504.13211/) (2025) - [Can You Share Your Story? Modeling Clients' Metacognition and Openness for LLM Therapist Evaluation](https://mlsanity.com/paper/2507.19643/) (2025) - [Enhancing Psychotherapy Counseling: A Data Augmentation Pipeline Leveraging Large Language Models for Counseling Conversations](https://mlsanity.com/paper/2406.08718/) (2024) - [RECAP: Resistance Capture in Text-based Mental Health Counseling with Large Language Models](https://mlsanity.com/paper/2601.14780/) (2026) - [Responsible Evaluation of AI for Mental Health](https://mlsanity.com/paper/2602.00065/) (2026) - [PanicToCalm: A Proactive Counseling Agent for Panic Attacks](https://mlsanity.com/paper/2510.21143/) (2025) - [Auditing Stealth Sycophancy in Mental-Health Dialogue: Structured Clinical-State Diagnostics and Clean Matched Benchmarks](https://mlsanity.com/paper/2605.03472/) (2026) - [When and How to Integrate Multimodal Large Language Models in College Psychotherapy: Perspectives from Multi-stakeholders](https://mlsanity.com/paper/2502.00229/) (2025) - ["Is This Really a Human Peer Supporter?": Misalignments Between Peer Supporters and Experts in LLM-Supported Interactions](https://mlsanity.com/paper/2506.09354/) (2025) - ["I Said Things I Needed to Hear Myself": Peer Support as an Emotional, Organisational, and Sociotechnical Practice in Singapore](https://mlsanity.com/paper/2506.09362/) (2025) - [Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs](https://mlsanity.com/paper/2602.21638/) (2026) - [Trust Stack for Mental Health AI: A Survey of Calibration across Human, Interaction, and AI Layers](https://mlsanity.com/paper/2604.20166/) (2026) - [Steering the Compass: Aligning Dynamic Psychological Counseling Conversations with Cognitive Behavioral Therapy Strategies](https://mlsanity.com/paper/2609.20565/) (2026) - [From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Health](https://mlsanity.com/paper/2609.25186/) (2026) ## Abstract (arXiv) Robust therapeutic relationships between counselors and clients are fundamental to counseling effectiveness. The assessment of therapeutic alliance is well-established in traditional face-to-face therapy but may not directly translate to text-based settings. With millions of individuals seeking support through online text-based counseling, understanding the relationship in such contexts is crucial. In this paper, we present an automatic approach using large language models (LLMs) to understand the development of therapeutic alliance in text-based counseling. We adapt a theoretically grounded framework specifically to the context of online text-based counseling and develop comprehensive guidelines for characterizing the alliance. We collect a comprehensive counseling dataset and conduct multiple expert evaluations on a subset based on this framework. Our LLM-based approach, combined with guidelines and simultaneous extraction of supportive evidence underlying its predictions, demonstrates effectiveness in identifying the therapeutic alliance. Through further LLM-based evaluations on additional conversations, our findings underscore the challenges counselors face in cultivating strong online relationships with clients. Furthermore, we demonstrate the potential of LLM-based feedback mechanisms to enhance counselors' ability to build relationships, supported by a small-scale proof-of-concept.