# Uncertainty Estimates for Efficient Neural Network-based Dialogue Policy Optimisation

arXiv: https://arxiv.org/abs/1711.11486 · HTML: https://mlsanity.com/paper/1711.11486/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2017.

## Built on by (most cited first, 2 of 2)

- [An Efficient Task-Oriented Dialogue Policy: Evolutionary Reinforcement Learning Injected by Elite Individuals](https://mlsanity.com/paper/2506.03519/) (2025)
- [On the Calibration and Uncertainty with P'olya-Gamma Augmentation for Dialog Retrieval Models](https://mlsanity.com/paper/2303.08606/) (2023)

## Abstract (arXiv)

In statistical dialogue management, the dialogue manager learns a policy that maps a belief state to an action for the system to perform. Efficient exploration is key to successful policy optimisation. Current deep reinforcement learning methods are very promising but rely on epsilon-greedy exploration, thus subjecting the user to a random choice of action during learning. Alternative approaches such as Gaussian Process SARSA (GPSARSA) estimate uncertainties and are sample efficient, leading to better user experience, but on the expense of a greater computational complexity. This paper examines approaches to extract uncertainty estimates from deep Q-networks (DQN) in the context of dialogue management. We perform an extensive benchmark of deep Bayesian methods to extract uncertainty estimates, namely Bayes-By-Backprop, dropout, its concrete variation, bootstrapped ensemble and alpha-divergences, combining it with DQN algorithm.