mlsanityData to 7 Oct 2026

A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning

arXiv 2202.13675 ↗First posted 28 Feb 2022cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

Dialogue Policy Learning is a key component in a task-oriented dialogue system (TDS) that decides the next action of the system given the dialogue state at each turn. Reinforcement Learning (RL) is commonly chosen to learn the dialogue policy, regarding the user as the environment and the system as the agent. Many benchmark datasets and algorithms have been created to facilitate the development and evaluation of dialogue policy based on RL. In this paper, we survey recent advances and challenges in dialogue policy from the prescriptive of RL. More specifically, we identify the major problems and summarize corresponding solutions for RL-based dialogue policy learning. Besides, we provide a comprehensive survey of applying RL to dialogue policy learning by categorizing recent methods into basic elements in RL. We believe this survey can shed a light on future research in dialogue management.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 67 of its 126 references that are in the map. First posted on arXiv 28 Feb 2022.

Builds on

67 of 126 references.

References · 67

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2016The Option-Critic Architecturecited by 433
  6. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
  7. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 202
  8. 2015A Neural Conversational Modelcited by 195
  9. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  10. 2018Neural Approaches to Conversational AIcited by 164
  11. 2017A Survey on Dialogue Systems: Recent Advances and New Frontierscited by 162
  12. 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 142
38 more
  1. 2015Neural Responding Machine for Short-Text Conversationcited by 124
  2. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 108
  3. 2019Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Contextcited by 99
  4. 2020Recent Advances and Challenges in Task-oriented Dialog Systemcited by 78
  5. 2019Dialog State Tracking: A Neural Reading Comprehension Approachcited by 62
  6. 2016A User Simulator for Task-Completion Dialoguescited by 56
  7. 2019Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Modelscited by 53
  8. 2017Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learningcited by 44
  9. 2019Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attentioncited by 43
  10. 2016A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systemscited by 39
  11. 2019Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialogcited by 37
  12. 2017BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systemscited by 30
  13. 2016Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learningcited by 29
  14. 2016On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systemscited by 28
  15. 2020Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue Systemcited by 28
  16. 2017Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Modelscited by 26
  17. 2020Multi-Domain Dialogue Acts and Response Co-Generationcited by 26
  18. 2019Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systemscited by 25
  19. 2020Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decompositioncited by 23
  20. 2018Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learningcited by 21
  21. 2020Is Your Goal-Oriented Dialog Model Performing Really Well? Empirical Analysis of System-wise Evaluationcited by 19
  22. 2017Sequence Modeling via Segmentationscited by 18
  23. 2017Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Managementcited by 18
  24. 2018Adversarial Learning of Task-Oriented Neural Dialog Modelscited by 17
  25. 2020Learning Goal-oriented Dialogue Policy with Opposite Agent Awarenesscited by 16
  26. 2018Sample Efficient Deep Reinforcement Learning for Dialogue Systems with Large Action Spacescited by 15
  27. 2016Policy Networks with Two-Stage Training for Dialogue Systemscited by 14
  28. 2017A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialoguecited by 14
  29. 2019Budgeted Policy Learning for Task-Oriented Dialogue Systemscited by 13
  30. 2016Continuously Learning Neural Dialogue Managementcited by 12
  31. 2018Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learningcited by 12
  32. 2018Goal-Oriented Chatbot Dialog Management Bootstrapping with Transfer Learningcited by 11
  33. 2018Feudal Reinforcement Learning for Dialogue Management in Large Domainscited by 11
  34. 2018Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learningcited by 11
  35. 2017Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learningcited by 10
  36. 2020Learning Dialog Policies from Weak Demonstrationscited by 10
  37. 2015Reward Shaping with Recurrent Neural Networks for Speeding up On-Line Policy Learning in Spoken Dialogue Systemscited by 9
  38. 2019Collaborative Multi-Agent Dialogue Model Training Via Reinforcement Learningcited by 9

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 6; 2025: 8; 2026: 4.

30 papers.

Sort:

By area:

  1. 2023Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agentscited by 76
  2. 2024A Dense Reward View on Aligning Text-to-Image Diffusion with Preferencecited by 55
  3. 2023A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Modelscited by 23
  4. 2023Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systemscited by 19
  5. 2023Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignmentcited by 19
  6. 2023End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directionscited by 17
  7. 2024Multi-User Chat Assistant (MUCA): a Framework Using LLMs to Facilitate Group Conversationscited by 13
  8. 2025Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?cited by 9
  9. 2023Towards Goal-oriented Intelligent Tutoring Systems in Online Educationcited by 7
  10. 2025Simulating Before Planning: Constructing Intrinsic User World Model for User-Tailored Dialogue Policy Planningcited by 7
  11. 2024Infusing Emotions into Task-oriented Dialogue Systems: Understanding, Management, and Generationcited by 5
  12. 2023TOD-Flow: Modeling the Structure of Task-Oriented Dialoguescited by 4
18 more
  1. 2025DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priorscited by 4
  2. 2024Rewarding What Matters: Step-by-Step Reinforcement Learning for Task-Oriented Dialoguecited by 3
  3. 2025An Efficient Task-Oriented Dialogue Policy: Evolutionary Reinforcement Learning Injected by Elite Individualscited by 2
  4. 2023Rescue Conversations from Dead-ends: Efficient Exploration for Task-oriented Dialogue Policy Optimizationcited by 1
  5. 2024Task Oriented Dialogue as a Catalyst for Self-Supervised Automatic Speech Recognitioncited by 1
  6. 2026One Model, Multiple Goals: Adaptive Multi-Objective Learning for E-commerce Dialogue Systemscited by 1
  7. 2023Why Guided Dialog Policy Learning performs well? Understanding the role of adversarial learning and its alternativecited by 0
  8. 2023JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learningcited by 0
  9. 2023Dialog Action-Aware Transformer for Dialog Policy Learningcited by 0
  10. 2023Learning Top-k Subtask Planning Tree based on Discriminative Representation Pre-training for Decision Makingcited by 0
  11. 2024Towards Zero-Shot, Controllable Dialog Planning with LLMscited by 0
  12. 2025Optimal Policy Adaptation under Covariate Shiftcited by 0
  13. 2025Policy Learning with a Natural Language Action Space: A Causal Approachcited by 0
  14. 2025Planning with Diffusion Models for Target-Oriented Dialogue Systemscited by 0
  15. 2025DyBBT: Dynamic Balance via Bandit-inspired Targeting for Dialog Policy with Cognitive Dual-Systemscited by 0
  16. 2026SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selectioncited by 0
  17. 2026Personality-Aware Reinforcement Learning for Persuasive Dialogue with LLM-Driven Simulationcited by 0
  18. 2026RPS: Information Elicitation with Reinforcement Prompt Selectioncited by 0