# A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning

arXiv: https://arxiv.org/abs/2202.13675 · HTML: https://mlsanity.com/paper/2202.13675/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 67 of its 126 references that are in the map.
- First posted on arXiv 28 Feb 2022.

## Builds on (most cited first, 30 of 67)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [Deal or No Deal? End-to-End Learning for Negotiation Dialogues](https://mlsanity.com/paper/1706.05125/) (2017)
- [A Neural Conversational Model](https://mlsanity.com/paper/1506.05869/) (2015)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [A Survey on Dialogue Systems: Recent Advances and New Frontiers](https://mlsanity.com/paper/1711.01731/) (2017)
- [Neural Belief Tracker: Data-Driven Dialogue State Tracking](https://mlsanity.com/paper/1606.03777/) (2016)
- [Neural Responding Machine for Short-Text Conversation](https://mlsanity.com/paper/1503.02364/) (2015)
- [A Laplacian Framework for Option Discovery in Reinforcement Learning](https://mlsanity.com/paper/1703.00956/) (2017)
- [Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context](https://mlsanity.com/paper/1911.10484/) (2019)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [Dialog State Tracking: A Neural Reading Comprehension Approach](https://mlsanity.com/paper/1908.01946/) (2019)
- [A User Simulator for Task-Completion Dialogues](https://mlsanity.com/paper/1612.05688/) (2016)
- [Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models](https://mlsanity.com/paper/1902.08858/) (2019)
- [Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning](https://mlsanity.com/paper/1704.03084/) (2017)
- [Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention](https://mlsanity.com/paper/1905.12866/) (2019)
- [A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems](https://mlsanity.com/paper/1607.00070/) (2016)
- [Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog](https://mlsanity.com/paper/1908.10719/) (2019)
- [BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1711.05715/) (2017)
- [Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning](https://mlsanity.com/paper/1606.02560/) (2016)
- [On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems](https://mlsanity.com/paper/1605.07669/) (2016)
- [Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System](https://mlsanity.com/paper/2006.06814/) (2020)
- [Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Models](https://mlsanity.com/paper/1709.06136/) (2017)
- [Multi-Domain Dialogue Acts and Response Co-Generation](https://mlsanity.com/paper/2004.12363/) (2020)
- [Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systems](https://mlsanity.com/paper/1905.05644/) (2019)

## Built on by (most cited first, 30 of 30)

- [Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents](https://mlsanity.com/paper/2311.00262/) (2023)
- [A Dense Reward View on Aligning Text-to-Image Diffusion with Preference](https://mlsanity.com/paper/2402.08265/) (2024)
- [A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Models](https://mlsanity.com/paper/2311.16789/) (2023)
- [Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systems](https://mlsanity.com/paper/2302.10342/) (2023)
- [Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment](https://mlsanity.com/paper/2310.08372/) (2023)
- [End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2311.09008/) (2023)
- [Multi-User Chat Assistant (MUCA): a Framework Using LLMs to Facilitate Group Conversations](https://mlsanity.com/paper/2401.04883/) (2024)
- [Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?](https://mlsanity.com/paper/2504.10000/) (2025)
- [Towards Goal-oriented Intelligent Tutoring Systems in Online Education](https://mlsanity.com/paper/2312.10053/) (2023)
- [Simulating Before Planning: Constructing Intrinsic User World Model for User-Tailored Dialogue Policy Planning](https://mlsanity.com/paper/2504.13643/) (2025)
- [Infusing Emotions into Task-oriented Dialogue Systems: Understanding, Management, and Generation](https://mlsanity.com/paper/2408.02417/) (2024)
- [TOD-Flow: Modeling the Structure of Task-Oriented Dialogues](https://mlsanity.com/paper/2312.04668/) (2023)
- [DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priors](https://mlsanity.com/paper/2505.17795/) (2025)
- [Rewarding What Matters: Step-by-Step Reinforcement Learning for Task-Oriented Dialogue](https://mlsanity.com/paper/2406.14457/) (2024)
- [An Efficient Task-Oriented Dialogue Policy: Evolutionary Reinforcement Learning Injected by Elite Individuals](https://mlsanity.com/paper/2506.03519/) (2025)
- [Rescue Conversations from Dead-ends: Efficient Exploration for Task-oriented Dialogue Policy Optimization](https://mlsanity.com/paper/2305.03262/) (2023)
- [Task Oriented Dialogue as a Catalyst for Self-Supervised Automatic Speech Recognition](https://mlsanity.com/paper/2401.02417/) (2024)
- [One Model, Multiple Goals: Adaptive Multi-Objective Learning for E-commerce Dialogue Systems](https://mlsanity.com/paper/2606.09293/) (2026)
- [Why Guided Dialog Policy Learning performs well? Understanding the role of adversarial learning and its alternative](https://mlsanity.com/paper/2307.06721/) (2023)
- [JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learning](https://mlsanity.com/paper/2309.00230/) (2023)
- [Dialog Action-Aware Transformer for Dialog Policy Learning](https://mlsanity.com/paper/2309.02240/) (2023)
- [Learning Top-k Subtask Planning Tree based on Discriminative Representation Pre-training for Decision Making](https://mlsanity.com/paper/2312.11027/) (2023)
- [Towards Zero-Shot, Controllable Dialog Planning with LLMs](https://mlsanity.com/paper/2410.05821/) (2024)
- [Optimal Policy Adaptation under Covariate Shift](https://mlsanity.com/paper/2501.08067/) (2025)
- [Policy Learning with a Natural Language Action Space: A Causal Approach](https://mlsanity.com/paper/2502.17538/) (2025)
- [Planning with Diffusion Models for Target-Oriented Dialogue Systems](https://mlsanity.com/paper/2504.16858/) (2025)
- [DyBBT: Dynamic Balance via Bandit-inspired Targeting for Dialog Policy with Cognitive Dual-Systems](https://mlsanity.com/paper/2509.19695/) (2025)
- [SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection](https://mlsanity.com/paper/2601.02871/) (2026)
- [Personality-Aware Reinforcement Learning for Persuasive Dialogue with LLM-Driven Simulation](https://mlsanity.com/paper/2601.06877/) (2026)
- [RPS: Information Elicitation with Reinforcement Prompt Selection](https://mlsanity.com/paper/2604.13817/) (2026)

## Abstract (arXiv)

Dialogue Policy Learning is a key component in a task-oriented dialogue system (TDS) that decides the next action of the system given the dialogue state at each turn. Reinforcement Learning (RL) is commonly chosen to learn the dialogue policy, regarding the user as the environment and the system as the agent. Many benchmark datasets and algorithms have been created to facilitate the development and evaluation of dialogue policy based on RL. In this paper, we survey recent advances and challenges in dialogue policy from the prescriptive of RL. More specifically, we identify the major problems and summarize corresponding solutions for RL-based dialogue policy learning. Besides, we provide a comprehensive survey of applying RL to dialogue policy learning by categorizing recent methods into basic elements in RL. We believe this survey can shed a light on future research in dialogue management.