# Taming Continuous Posteriors for Latent Variational Dialogue Policies

arXiv: https://arxiv.org/abs/2205.07633 · HTML: https://mlsanity.com/paper/2205.07633/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 16 May 2022.

## Builds on (most cited first, 30 of 30)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders](https://mlsanity.com/paper/1703.10960/) (2017)
- [Deal or No Deal? End-to-End Learning for Negotiation Dialogues](https://mlsanity.com/paper/1706.05125/) (2017)
- [A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues](https://mlsanity.com/paper/1605.06069/) (2016)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [Unsupervised Evaluation of Interactive Dialog with DialoGPT](https://mlsanity.com/paper/2006.12719/) (2020)
- [Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context](https://mlsanity.com/paper/1911.10484/) (2019)
- [Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog](https://mlsanity.com/paper/1706.08502/) (2017)
- [Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models](https://mlsanity.com/paper/1902.08858/) (2019)
- [Training Neural Response Selection for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1906.01543/) (2019)
- [DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder](https://mlsanity.com/paper/1805.12352/) (2018)
- [Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention](https://mlsanity.com/paper/1905.12866/) (2019)
- [CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackers](https://mlsanity.com/paper/2010.12850/) (2020)
- [Structured Fusion Networks for Dialog](https://mlsanity.com/paper/1907.10016/) (2019)
- [Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning](https://mlsanity.com/paper/1702.03274/) (2017)
- [LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization](https://mlsanity.com/paper/2011.09378/) (2020)
- [Hierarchical Reinforcement Learning for Open-Domain Dialog](https://mlsanity.com/paper/1909.07547/) (2019)
- [Multi-Domain Dialogue Acts and Response Co-Generation](https://mlsanity.com/paper/2004.12363/) (2020)
- [UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues](https://mlsanity.com/paper/2004.14307/) (2020)
- [Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management](https://mlsanity.com/paper/1707.00130/) (2017)
- [A Hierarchical Latent Structure for Variational Conversation Modeling](https://mlsanity.com/paper/1804.03424/) (2018)
- [Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach](https://mlsanity.com/paper/2102.10242/) (2021)
- [Policy Networks with Two-Stage Training for Dialogue Systems](https://mlsanity.com/paper/1606.03152/) (2016)
- [Improving Variational Encoder-Decoders in Dialogue Generation](https://mlsanity.com/paper/1802.02032/) (2018)
- [Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts](https://mlsanity.com/paper/1807.07255/) (2018)
- [SUMBT+LaRL: Effective Multi-domain End-to-end Neural Task-oriented Dialog System](https://mlsanity.com/paper/2009.10447/) (2020)

## Built on by (most cited first, 1 of 1)

- [Learning from Relevant Subgoals in Successful Dialogs using Iterative Training for Task-oriented Dialog Systems](https://mlsanity.com/paper/2411.16305/) (2024)

## Abstract (arXiv)

Utilizing amortized variational inference for latent-action reinforcement learning (RL) has been shown to be an effective approach in Task-oriented Dialogue (ToD) systems for optimizing dialogue success. Until now, categorical posteriors have been argued to be one of the main drivers of performance. In this work we revisit Gaussian variational posteriors for latent-action RL and show that they can yield even better performance than categoricals. We achieve this by simplifying the training procedure and propose ways to regularize the latent dialogue policy to retain good response coherence. Using continuous latent representations our model achieves state of the art dialogue success rate on the MultiWOZ benchmark, and also compares well to categorical latent methods in response coherence.