mlsanityData to 7 Oct 2026

Deep Reinforcement Learning for Dialogue Generation

arXiv 1606.01541 ↗First posted 5 Jun 2016cs.CLcited by 286

Not on the map. Why?

Cited by 25 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent neural models of dialogue generation offer great promise for generating responses for conversational agents, but tend to be shortsighted, predicting utterances one at a time while ignoring their influence on future outcomes. Modeling the future direction of a dialogue is crucial to generating coherent, interesting dialogues, a need which led traditional NLP models of dialogue to draw on reinforcement learning. In this paper, we show how to integrate these goals, applying deep reinforcement learning to model future reward in chatbot dialogue. The model simulates dialogues between two virtual agents, using policy gradient methods to reward sequences that display three useful conversational properties: informativity (non-repetitive turns), coherence, and ease of answering (related to forward-looking function). We evaluate our model on diversity, length as well as with human judges, showing that the proposed algorithm generates more interactive responses and manages to foster a more sustained conversation in dialogue simulation. This work marks a first step towards learning a neural conversational model based on the long-term success of dialogues.

Key facts. Cited by 286 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 1; 2018: 1; 2019: 1; 2020: 2; 2021: 85; 2022: 62; 2023: 64; 2024: 30; 2025: 25; 2026: 15.

286 papers.

Sort:

By area:

  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  2. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  3. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  4. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  5. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
  6. 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 184
  7. 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 172
  8. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  9. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  10. 2022Calibrating Sequence likelihood Improves Conditional Language Generationcited by 121
  11. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  12. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
38 more
  1. 2023f-Divergence Minimization for Sequence-Level Knowledge Distillationcited by 112
  2. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  3. 2022Long Time No See! Open-Domain Conversation with Long-Term Persona Memorycited by 107
  4. 2023Guided Flows for Generative Modeling and Decision Makingcited by 99
  5. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 94
  6. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 92
  7. 2022The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systemscited by 92
  8. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 84
  9. 2022The Role of Coverage in Online Reinforcement Learningcited by 82
  10. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80
  11. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  12. 2021Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approachcited by 75
  13. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 75
  14. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedbackcited by 71
  15. 2024Multi-turn Reinforcement Learning from Preference Human Feedbackcited by 65
  16. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Modelcited by 56
  17. 2022LENS: A Learnable Evaluation Metric for Text Simplificationcited by 54
  18. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 53
  19. 2023Learning to Generate Better Than Your LLMcited by 49
  20. 2022Critic-Guided Decoding for Controlled Text Generationcited by 45
  21. 2022Neural Approaches to Conversational Information Retrievalcited by 41
  22. 2023Reinforcement Learning-based Counter-Misinformation Response Generation: A Case Study of COVID-19 Vaccine Misinformationcited by 39
  23. 2021PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generationcited by 37
  24. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
  25. 2021Efficient (Soft) Q-Learning for Text Generation with Limited Good Datacited by 35
  26. 2022Dungeons and Dragons as a Dialog Challenge for Artificial Intelligencecited by 35
  27. 2021Gradient Imitation Reinforcement Learning for Low Resource Relation Extractioncited by 33
  28. 2022CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learningcited by 33
  29. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  30. 2021Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 32
  31. 2022On the Complexity of Adversarial Decision Makingcited by 32
  32. 2021Survey on reinforcement learning for language processingcited by 31
  33. 2022Off-Policy Confidence Interval Estimation with Confounded Markov Decision Processcited by 31
  34. 2022Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbotscited by 31
  35. 2021Defending Against Backdoor Attacks in Natural Language Generationcited by 30
  36. 2021Neural Collaborative Graph Machines for Table Structure Recognitioncited by 30
  37. 2022Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Studycited by 30
  38. 2023Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue Systemcited by 30

50 most cited of 286 All 286 (JSON)