mlsanityData to 14 Oct 2026

Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning

arXiv 2208.02294 ↗First posted 25 Jul 2022cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Despite recent advances in natural language understanding and generation, and decades of research on the development of conversational bots, building automated agents that can carry on rich open-ended conversations with humans "in the wild" remains a formidable challenge. In this work we develop a real-time, open-ended dialogue system that uses reinforcement learning (RL) to power a bot's conversational skill at scale. Our work pairs the succinct embedding of the conversation state generated using SOTA (supervised) language models with RL techniques that are particularly suited to a dynamic action space that changes as the conversation progresses. Trained using crowd-sourced data, our novel system is able to substantially exceeds the (strong) baseline supervised model with respect to several metrics of interest in a live experiment with real users of the Google Assistant.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 24 of its 48 references that are in the map. First posted on arXiv 25 Jul 2022.

Builds on

24 of 48 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  5. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  6. 2018Universal Sentence Encodercited by 489
  7. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
  8. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 284
  9. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  10. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 173
  11. 2018The Design and Implementation of XiaoIce, an Empathetic Social Chatbotcited by 172
  12. 2017Adversarial Learning for Neural Dialogue Generationcited by 103
12 more
  1. 2019Encode, Tag, Realize: High-Precision Text Editingcited by 99
  2. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 93
  3. 2018Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systemscited by 53
  4. 2019Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Modelscited by 53
  5. 2017A Deep Reinforcement Learning Chatbotcited by 36
  6. 2019DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusioncited by 29
  7. 2019Generating Empathetic Responses by Looking Ahead the User's Sentimentcited by 27
  8. 2019Hierarchical Reinforcement Learning for Open-Domain Dialogcited by 27
  9. 2019CAQL: Continuous Action Q-Learningcited by 23
  10. 2016Policy Networks with Two-Stage Training for Dialogue Systemscited by 14
  11. 2018Planning and Learning with Stochastic Action Setscited by 9
  12. 2016Sample-efficient Deep Reinforcement Learning for Dialog Controlcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 2.

8 papers.

  1. 2023Augmented Language Models: a Surveycited by 403
  2. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 280
  3. 2023Challenges and Applications of Large Language Modelscited by 254
  4. 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
  5. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 12
  6. 2023Transformers in Reinforcement Learning: A Surveycited by 12
  7. 2024Locating and Mitigating Gender Bias in Large Language Modelscited by 12
  8. 2024SAPIENT: Mastering Multi-turn Conversational Recommendation with Strategic Planning and Monte Carlo Tree Searchcited by 6