# Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach

arXiv: https://arxiv.org/abs/2101.07714 · HTML: https://mlsanity.com/paper/2101.07714/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 96 references that are in the map.
- First posted on arXiv 19 Jan 2021.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support](https://mlsanity.com/paper/2009.08441/) (2020)
- [Challenges in Building Intelligent Open-domain Dialog Systems](https://mlsanity.com/paper/1905.05709/) (2019)
- [What makes a good conversation? How controllable attributes affect human judgments](https://mlsanity.com/paper/1902.08654/) (2019)
- [MIME: MIMicking Emotions for Empathetic Response Generation](https://mlsanity.com/paper/2010.01454/) (2020)
- [Automatically Neutralizing Subjective Bias in Text](https://mlsanity.com/paper/1911.09709/) (2019)
- [Gmail Smart Compose: Real-Time Assisted Writing](https://mlsanity.com/paper/1906.00080/) (2019)
- [MoEL: Mixture of Empathetic Listeners](https://mlsanity.com/paper/1908.07687/) (2019)
- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019)
- [A Probabilistic Formulation of Unsupervised Text Style Transfer](https://mlsanity.com/paper/2002.03912/) (2020)
- [A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer](https://mlsanity.com/paper/1905.10060/) (2019)
- [PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction](https://mlsanity.com/paper/2010.13816/) (2020)
- [Modeling Empathy and Distress in Reaction to News Stories](https://mlsanity.com/paper/1808.10399/) (2018)
- [Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity](https://mlsanity.com/paper/1809.06873/) (2018)
- [A Dynamic Strategy Coach for Effective Negotiation](https://mlsanity.com/paper/1909.13426/) (2019)
- [Plug and Play Autoencoders for Conditional Text Generation](https://mlsanity.com/paper/2010.02983/) (2020)
- [Balancing Objectives in Counseling Conversations: Advancing Forwards or Looking Backwards](https://mlsanity.com/paper/2005.04245/) (2020)

## Built on by (most cited first, 30 of 75)

- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods](https://mlsanity.com/paper/2404.00282/) (2024)
- [SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents](https://mlsanity.com/paper/2403.08715/) (2024)
- [Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction](https://mlsanity.com/paper/2305.02466/) (2023)
- [Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Support](https://mlsanity.com/paper/2203.15144/) (2022)
- [Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data](https://mlsanity.com/paper/2307.14385/) (2023)
- [Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring](https://mlsanity.com/paper/2310.15461/) (2023)
- [Critic-Guided Decoding for Controlled Text Generation](https://mlsanity.com/paper/2212.10938/) (2022)
- [CoMAE: A Multi-factor Hierarchical Framework for Empathetic Response Generation](https://mlsanity.com/paper/2105.08316/) (2021)
- [Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversations](https://mlsanity.com/paper/2305.10172/) (2023)
- [Benefits and Harms of Large Language Models in Digital Mental Health](https://mlsanity.com/paper/2311.14693/) (2023)
- [Reinforcement Learning-based Counter-Misinformation Response Generation: A Case Study of COVID-19 Vaccine Misinformation](https://mlsanity.com/paper/2303.06433/) (2023)
- [Evidence-Driven Retrieval Augmented Response Generation for Online Misinformation](https://mlsanity.com/paper/2403.14952/) (2024)
- [Helping the Helper: Supporting Peer Counselors via AI-Empowered Practice and Feedback](https://mlsanity.com/paper/2305.08982/) (2023)
- [When Large Language Models are Reliable for Judging Empathic Communication](https://mlsanity.com/paper/2506.10150/) (2025)
- [Empathetic Dialogue Generation via Sensitive Emotion Recognition and Sensible Knowledge Selection](https://mlsanity.com/paper/2210.11715/) (2022)
- [Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networks](https://mlsanity.com/paper/2302.05675/) (2023)
- [Gendered Mental Health Stigma in Masked Language Models](https://mlsanity.com/paper/2210.15144/) (2022)
- [Towards an Online Empathetic Chatbot with Emotion Causes](https://mlsanity.com/paper/2105.11903/) (2021)
- [Rethinking Large Language Models in Mental Health Applications](https://mlsanity.com/paper/2311.11267/) (2023)
- [Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunities](https://mlsanity.com/paper/2206.05017/) (2022)
- [CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy](https://mlsanity.com/paper/2410.13218/) (2024)
- [Knowledge Planning in Large Language Models for Domain-Aligned Counseling Summarization](https://mlsanity.com/paper/2409.14907/) (2024)
- [Transferable Persona-Grounded Dialogues via Grounded Minimal Edits](https://mlsanity.com/paper/2109.07713/) (2021)
- [A Cognitive Stimulation Dialogue System with Multi-source Knowledge Fusion for Elders with Cognitive Impairment](https://mlsanity.com/paper/2305.08200/) (2023)
- [Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation](https://mlsanity.com/paper/2403.13578/) (2024)
- [RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions](https://mlsanity.com/paper/2601.13235/) (2026)
- [Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication](https://mlsanity.com/paper/2106.11791/) (2021)

## Abstract (arXiv)

Online peer-to-peer support platforms enable conversations between millions of people who seek and provide mental health support. If successful, web-based mental health conversations could improve access to treatment and reduce the global disease burden. Psychologists have repeatedly demonstrated that empathy, the ability to understand and feel the emotions and experiences of others, is a key component leading to positive outcomes in supportive conversations. However, recent studies have shown that highly empathic conversations are rare in online mental health platforms. In this paper, we work towards improving empathy in online mental health support conversations. We introduce a new task of empathic rewriting which aims to transform low-empathy conversational posts to higher empathy. Learning such transformations is challenging and requires a deep understanding of empathy while maintaining conversation quality through text fluency and specificity to the conversational context. Here we propose PARTNER, a deep reinforcement learning agent that learns to make sentence-level edits to posts in order to increase the expressed level of empathy while maintaining conversation quality. Our RL agent leverages a policy network, based on a transformer language model adapted from GPT-2, which performs the dual task of generating candidate empathic sentences and adding those sentences at appropriate positions. During training, we reward transformations that increase empathy in posts while maintaining text fluency, context specificity and diversity. Through a combination of automatic and human evaluation, we demonstrate that PARTNER successfully generates more empathic, specific, and diverse responses and outperforms NLP methods from related tasks like style transfer and empathic dialogue generation. Our work has direct implications for facilitating empathic conversations on web-based platforms.