# Deep Reinforcement Learning For Modeling Chit-Chat Dialog With Discrete Attributes

arXiv: https://arxiv.org/abs/1907.02848 · HTML: https://mlsanity.com/paper/1907.02848/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 11 of 11)

- [Quark: Controllable Text Generation with Reinforced Unlearning](https://mlsanity.com/paper/2205.13636/) (2022)
- [Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation](https://mlsanity.com/paper/2106.08942/) (2021)
- [Survey on reinforcement learning for language processing](https://mlsanity.com/paper/2104.05565/) (2021)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Multi-User Chat Assistant (MUCA): a Framework Using LLMs to Facilitate Group Conversations](https://mlsanity.com/paper/2401.04883/) (2024)
- [Towards Standard Criteria for human evaluation of Chatbots: A Survey](https://mlsanity.com/paper/2105.11197/) (2021)
- [Zero-Shot Controlled Generation with Encoder-Decoder Transformers](https://mlsanity.com/paper/2106.06411/) (2021)
- [Controllable Dialogue Generation with Disentangled Multi-grained Style Specification and Attribute Consistency Reward](https://mlsanity.com/paper/2109.06717/) (2021)
- [CORAL: Contextual Response Retrievability Loss Function for Training Dialog Generation Models](https://mlsanity.com/paper/2205.10558/) (2022)
- [Use of a Taxonomy of Empathetic Response Intents to Control and Interpret Empathy in Neural Chatbots](https://mlsanity.com/paper/2305.10096/) (2023)
- [Policy-driven Knowledge Selection and Response Generation for Document-grounded Dialogue](https://mlsanity.com/paper/2410.15970/) (2024)

## Abstract (arXiv)

Open domain dialog systems face the challenge of being repetitive and producing generic responses. In this paper, we demonstrate that by conditioning the response generation on interpretable discrete dialog attributes and composed attributes, it helps improve the model perplexity and results in diverse and interesting non-redundant responses. We propose to formulate the dialog attribute prediction as a reinforcement learning (RL) problem and use policy gradients methods to optimize utterance generation using long-term rewards. Unlike existing RL approaches which formulate the token prediction as a policy, our method reduces the complexity of the policy optimization by limiting the action space to dialog attributes, thereby making the policy optimization more practical and sample efficient. We demonstrate this with experimental and human evaluations.