# Context-Emotion Aware Therapeutic Dialogue Generation: A Multi-component Reinforcement Learning Approach to Language Models for Mental Health Support

arXiv: https://arxiv.org/abs/2511.11884 · HTML: https://mlsanity.com/paper/2511.11884/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 14 Nov 2025.

## Builds on (most cited first, 20 of 20)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Towards Emotional Support Dialog Systems](https://mlsanity.com/paper/2106.01144/) (2021)
- [Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking](https://mlsanity.com/paper/2312.09244/) (2023)
- [Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach](https://mlsanity.com/paper/2101.07714/) (2021)
- [Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support](https://mlsanity.com/paper/2307.15810/) (2023)
- [The opportunities and risks of large language models in mental health](https://mlsanity.com/paper/2403.14814/) (2024)
- [EmoLLM: Multimodal Emotional Understanding Meets Large Language Models](https://mlsanity.com/paper/2406.16442/) (2024)
- [The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support](https://mlsanity.com/paper/2401.14362/) (2024)
- [Towards Multimodal Emotional Support Conversation Systems](https://mlsanity.com/paper/2408.03650/) (2024)
- [Can AI Relate: Testing Large Language Model Response for Mental Health Support](https://mlsanity.com/paper/2405.12021/) (2024)
- [PsyEval: A Suite of Mental Health Related Tasks for Evaluating Large Language Models](https://mlsanity.com/paper/2311.09189/) (2023)
- [Fine-Tuning Language Models with Reward Learning on Policy](https://mlsanity.com/paper/2403.19279/) (2024)
- [Large Language Model for Mental Health: A Systematic Review](https://mlsanity.com/paper/2403.15401/) (2024)
- [Assessing Empathy in Large Language Models with Real-World Physician-Patient Interactions](https://mlsanity.com/paper/2405.16402/) (2024)
- [Clinical Insights: A Comprehensive Review of Language Models in Medicine](https://mlsanity.com/paper/2408.11735/) (2024)

## Abstract (arXiv)

Mental health disorders impose a substantial global socioeconomic burden. While large language models (LLMs) offer 24/7, non-judgmental interactions to address this gap, pretrained models lack contextual coherence and emotional alignment for appropriate therapeutic dialogue. Existing methods suffer from three critical methodological gaps: 1) Supervised Fine-Tuning (SFT) produces repetitive, context-insensitive outputs that fail to balance clinical accuracy with genuine empathy; 2) Reinforcement Learning (RL)-based therapeutic systems rely on generic reward functions (e.g., BLEU, ROUGE) that prioritise lexical similarity over clinical-specific emotional appropriateness and contextual relevance; 3) LLMs are resource-intensive and pose data privacy risks, making local deployment in clinical settings infeasible. To address these gaps, this study investigates the application of SFT and RL techniques to enhance GPT-2's capacity for therapeutic dialogue generation. The methodology restructured input formats to enable simultaneous processing of contextual information and emotional states alongside user input, employing a novel multi-component reward function that explicitly aligns model outputs with professional therapeutic logic (not just lexical overlap) and annotated emotions. Results demonstrated substantial improvements through RLs over baseline GPT-2 across multiple evaluation metrics: BLEU (0.0111), ROUGE-1 (0.1397), ROUGE-2 (0.0213), ROUGE-L (0.1317), and METEOR (0.0581). LLM evaluation confirmed high contextual relevance and professionalism, while RL achieved 99.34% emotion accuracy compared to 66.96% for baseline GPT-2. These findings demonstrate RL's effectiveness in developing therapeutic dialogue systems that can serve as valuable assistive tools for therapists, while maintaining essential human clinical oversight.