# Response-act Guided Reinforced Dialogue Generation for Mental Health Counseling

arXiv: https://arxiv.org/abs/2301.12729 · HTML: https://mlsanity.com/paper/2301.12729/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://mlsanity.com/paper/1507.04808/) (2015)
- [TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents](https://mlsanity.com/paper/1901.08149/) (2019)
- [The Design and Implementation of XiaoIce, an Empathetic Social Chatbot](https://mlsanity.com/paper/1812.08989/) (2018)
- [A Distributional Approach to Controlled Text Generation](https://mlsanity.com/paper/2012.11635/) (2020)
- [UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2](https://mlsanity.com/paper/2012.03539/) (2020)
- [Target-Guided Open-Domain Conversation](https://mlsanity.com/paper/1905.11553/) (2019)
- [Structured Fusion Networks for Dialog](https://mlsanity.com/paper/1907.10016/) (2019)
- [DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation](https://mlsanity.com/paper/2204.13031/) (2022)
- [ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation](https://mlsanity.com/paper/2104.08006/) (2021)
- [Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversations](https://mlsanity.com/paper/2111.06647/) (2021)
- [Knowledge-Aware Graph-Enhanced GPT-2 for Dialogue State Tracking](https://mlsanity.com/paper/2104.04466/) (2021)
- [A Hierarchical Latent Structure for Variational Conversation Modeling](https://mlsanity.com/paper/1804.03424/) (2018)
- [Neural Response Generation with Meta-Words](https://mlsanity.com/paper/1906.06050/) (2019)
- [The workweek is the best time to start a family -- A Study of GPT-2 Based Claim Generation](https://mlsanity.com/paper/2010.06185/) (2020)
- [Learning from Real Users: Rating Dialogue Success with Neural Networks for Reinforcement Learning in Spoken Dialogue Systems](https://mlsanity.com/paper/1508.03386/) (2015)
- [Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentation](https://mlsanity.com/paper/2205.09314/) (2022)
- [Retrospective and Prospective Mixture-of-Generators for Task-oriented Dialogue Response Generation](https://mlsanity.com/paper/1911.08151/) (2019)
- [Diversifying Task-oriented Dialogue Response Generation with Prototype Guided Paraphrasing](https://mlsanity.com/paper/2008.03391/) (2020)
- [Effective Incorporation of Speaker Information in Utterance Encoding in Dialog](https://mlsanity.com/paper/1907.05599/) (2019)
- [Prompt Tuning GPT-2 language model for parameter-efficient domain adaptation of ASR systems](https://mlsanity.com/paper/2112.08718/) (2021)

## Built on by (most cited first, 9 of 9)

- [Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring](https://mlsanity.com/paper/2310.15461/) (2023)
- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Natural Language Generation in Healthcare: A Review of Methods and Applications](https://mlsanity.com/paper/2505.04073/) (2025)
- [Redefining Experts: Interpretable Decomposition of Language Models for Toxicity Mitigation](https://mlsanity.com/paper/2509.16660/) (2025)
- [Data Augmentation of Multi-turn Psychological Dialogue via Knowledge-driven Progressive Thought Prompting](https://mlsanity.com/paper/2406.16567/) (2024)
- [Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems](https://mlsanity.com/paper/2307.07255/) (2023)
- [coTherapist: A Behavior-Aligned Small Language Model to Support Mental Healthcare Experts](https://mlsanity.com/paper/2601.10246/) (2026)
- [Mentalic Net: Development of RAG-based Conversational AI and Evaluation Framework for Mental Health Support](https://mlsanity.com/paper/2509.04456/) (2025)

## Abstract (arXiv)

Virtual Mental Health Assistants (VMHAs) have become a prevalent method for receiving mental health counseling in the digital healthcare space. An assistive counseling conversation commences with natural open-ended topics to familiarize the client with the environment and later converges into more fine-grained domain-specific topics. Unlike other conversational systems, which are categorized as open-domain or task-oriented systems, VMHAs possess a hybrid conversational flow. These counseling bots need to comprehend various aspects of the conversation, such as dialogue-acts, intents, etc., to engage the client in an effective conversation. Although the surge in digital health research highlights applications of many general-purpose response generation systems, they are barely suitable in the mental health domain -- the prime reason is the lack of understanding in mental health counseling. Moreover, in general, dialogue-act guided response generators are either limited to a template-based paradigm or lack appropriate semantics. To this end, we propose READER -- a REsponse-Act guided reinforced Dialogue genERation model for the mental health counseling conversations. READER is built on transformer to jointly predict a potential dialogue-act d(t+1) for the next utterance (aka response-act) and to generate an appropriate response u(t+1). Through the transformer-reinforcement-learning (TRL) with Proximal Policy Optimization (PPO), we guide the response generator to abide by d(t+1) and ensure the semantic richness of the responses via BERTScore in our reward computation. We evaluate READER on HOPE, a benchmark counseling conversation dataset and observe that it outperforms several baselines across several evaluation metrics -- METEOR, ROUGE, and BERTScore. We also furnish extensive qualitative and quantitative analyses on results, including error analysis, human evaluation, etc.