# Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance

arXiv: https://arxiv.org/abs/2109.04349 · HTML: https://mlsanity.com/paper/2109.04349/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 58 references that are in the map.
- First posted on arXiv 9 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Ensemble Distribution Distillation](https://mlsanity.com/paper/1905.00076/) (2019)
- [Neural Belief Tracker: Data-Driven Dialogue State Tracking](https://mlsanity.com/paper/1606.03777/) (2016)
- [TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking](https://mlsanity.com/paper/2005.02877/) (2020)
- [Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking](https://mlsanity.com/paper/1910.03544/) (2019)
- [SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking](https://mlsanity.com/paper/1907.07421/) (2019)
- [ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems](https://mlsanity.com/paper/2002.04793/) (2020)
- [CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackers](https://mlsanity.com/paper/2010.12850/) (2020)
- [BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1907.03040/) (2019)
- [Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog](https://mlsanity.com/paper/1908.10719/) (2019)
- [Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks](https://mlsanity.com/paper/2101.10642/) (2021)
- [Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing](https://mlsanity.com/paper/1807.06517/) (2018)
- [Slot Self-Attentive Dialogue State Tracking](https://mlsanity.com/paper/2101.09374/) (2021)
- [Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets](https://mlsanity.com/paper/2105.06987/) (2021)
- [Sample Efficient Deep Reinforcement Learning for Dialogue Systems with Large Action Spaces](https://mlsanity.com/paper/1802.03753/) (2018)
- [Distributed Structured Actor-Critic Reinforcement Learning for Universal Dialogue Management](https://mlsanity.com/paper/2009.10326/) (2020)
- [Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity](https://mlsanity.com/paper/1911.03700/) (2019)
- [Knowing What You Know: Calibrating Dialogue Belief State Distributions via Ensembles](https://mlsanity.com/paper/2010.02586/) (2020)

## Built on by (most cited first, 8 of 8)

- ["Do you follow me?": A Survey of Recent Approaches in Dialogue State Tracking](https://mlsanity.com/paper/2207.14627/) (2022)
- [ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format](https://mlsanity.com/paper/2211.17148/) (2022)
- [Dynamic Dialogue Policy for Continual Reinforcement Learning](https://mlsanity.com/paper/2204.05928/) (2022)
- [Infusing Emotions into Task-oriented Dialogue Systems: Understanding, Management, and Generation](https://mlsanity.com/paper/2408.02417/) (2024)
- [Confidence Estimation for LLM-Based Dialogue State Tracking](https://mlsanity.com/paper/2409.09629/) (2024)
- [From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue](https://mlsanity.com/paper/2308.12648/) (2023)
- [A Confidence-based Acquisition Model for Self-supervised Active Learning and Label Correction](https://mlsanity.com/paper/2310.08944/) (2023)
- [All Learning Has an Emotional Basis, So Does Task-Oriented Dialogue](https://mlsanity.com/paper/2507.01594/) (2025)

## Abstract (arXiv)

The ability to identify and resolve uncertainty is crucial for the robustness of a dialogue system. Indeed, this has been confirmed empirically on systems that utilise Bayesian approaches to dialogue belief tracking. However, such systems consider only confidence estimates and have difficulty scaling to more complex settings. Neural dialogue systems, on the other hand, rarely take uncertainties into account. They are therefore overconfident in their decisions and less robust. Moreover, the performance of the tracking task is often evaluated in isolation, without consideration of its effect on the downstream policy optimisation. We propose the use of different uncertainty measures in neural belief tracking. The effects of these measures on the downstream task of policy optimisation are evaluated by adding selected measures of uncertainty to the feature space of the policy and training policies through interaction with a user simulator. Both human and simulated user results show that incorporating these measures leads to improvements both of the performance and of the robustness of the downstream dialogue policy. This highlights the importance of developing neural dialogue belief trackers that take uncertainty into account.