# Sample-efficient Deep Reinforcement Learning for Dialog Control

arXiv: https://arxiv.org/abs/1612.06000 · HTML: https://mlsanity.com/paper/1612.06000/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2016.

## Built on by (most cited first, 5 of 5)

- [A Mixture-of-Expert Approach to RL-based Dialogue Management](https://mlsanity.com/paper/2206.00059/) (2022)
- [Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning](https://mlsanity.com/paper/2208.02294/) (2022)
- [Reduced Robust Random Cut Forest for Out-Of-Distribution detection in machine learning models](https://mlsanity.com/paper/2206.09247/) (2022)
- [Factual and Personalized Recommendations using Language Models and Reinforcement Learning](https://mlsanity.com/paper/2310.06176/) (2023)
- [Offline Reinforcement Learning for Mixture-of-Expert Dialogue Management](https://mlsanity.com/paper/2302.10850/) (2023)

## Abstract (arXiv)

Representing a dialog policy as a recurrent neural network (RNN) is attractive because it handles partial observability, infers a latent representation of state, and can be optimized with supervised learning (SL) or reinforcement learning (RL). For RL, a policy gradient approach is natural, but is sample inefficient. In this paper, we present 3 methods for reducing the number of dialogs required to optimize an RNN-based dialog policy with RL. The key idea is to maintain a second RNN which predicts the value of the current policy, and to apply experience replay to both networks. On two tasks, these methods reduce the number of dialogs/episodes required by about a third, vs. standard policy gradient methods.