# LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization

arXiv: https://arxiv.org/abs/2011.09378 · HTML: https://mlsanity.com/paper/2011.09378/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2020.

## Built on by (most cited first, 30 of 30)

- [GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection](https://mlsanity.com/paper/2111.14592/) (2021)
- [Shades of BLEU, Flavours of Success: The Case of MultiWOZ](https://mlsanity.com/paper/2106.05555/) (2021)
- [SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation](https://mlsanity.com/paper/2209.06664/) (2022)
- [AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models](https://mlsanity.com/paper/2102.05126/) (2021)
- [ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format](https://mlsanity.com/paper/2211.17148/) (2022)
- [BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialog](https://mlsanity.com/paper/2205.02471/) (2022)
- [End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2311.09008/) (2023)
- [Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialogues](https://mlsanity.com/paper/2412.14584/) (2024)
- [Variational Latent-State GPT for Semi-Supervised Task-Oriented Dialog Systems](https://mlsanity.com/paper/2109.04314/) (2021)
- [KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learning](https://mlsanity.com/paper/2211.16773/) (2022)
- [Retrieve & Memorize: Dialog Policy Learning with Multi-Action Memory](https://mlsanity.com/paper/2106.02317/) (2021)
- [NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural Language Understanding in Task-Oriented Dialogue](https://mlsanity.com/paper/2204.13021/) (2022)
- [Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goals](https://mlsanity.com/paper/2309.08949/) (2023)
- [A Generative User Simulator with GPT-based Architecture and Goal State Tracking for Reinforced Multi-Domain Dialog Systems](https://mlsanity.com/paper/2210.08692/) (2022)
- [DiactTOD: Learning Generalizable Latent Dialogue Acts for Controllable Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2308.00878/) (2023)
- [InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2310.08885/) (2023)
- [Causal-aware Safe Policy Improvement for Task-oriented dialogue](https://mlsanity.com/paper/2103.06370/) (2021)
- [OmniDialog: An Omnipotent Pre-training Model for Task-Oriented Dialogue System](https://mlsanity.com/paper/2312.16864/) (2023)
- [What Does The User Want? Information Gain for Hierarchical Dialogue Policy Optimisation](https://mlsanity.com/paper/2109.07129/) (2021)
- [Dialogue Evaluation with Offline Reinforcement Learning](https://mlsanity.com/paper/2209.00876/) (2022)
- [Learning Interpretable Latent Dialogue Actions With Less Supervision](https://mlsanity.com/paper/2209.11128/) (2022)
- [DORA: Toward Policy Optimization for Task-oriented Dialogue System with Efficient Context](https://mlsanity.com/paper/2107.03286/) (2021)
- [Contextual Data Augmentation for Task-Oriented Dialog Systems](https://mlsanity.com/paper/2310.10380/) (2023)
- [All Learning Has an Emotional Basis, So Does Task-Oriented Dialogue](https://mlsanity.com/paper/2507.01594/) (2025)
- [Taming Continuous Posteriors for Latent Variational Dialogue Policies](https://mlsanity.com/paper/2205.07633/) (2022)
- [UBARv2: Towards Mitigating Exposure Bias in Task-Oriented Dialogs](https://mlsanity.com/paper/2209.07239/) (2022)
- [Replicating Complex Dialogue Policy of Humans via Offline Imitation Learning with Supervised Regularization](https://mlsanity.com/paper/2305.03987/) (2023)
- [Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking](https://mlsanity.com/paper/2312.01842/) (2023)
- [Learning from Relevant Subgoals in Successful Dialogs using Iterative Training for Task-oriented Dialog Systems](https://mlsanity.com/paper/2411.16305/) (2024)
- [Collaborative and Proactive Management of Task-Oriented Conversations](https://mlsanity.com/paper/2510.05110/) (2025)

## Abstract (arXiv)

Reinforcement learning (RL) can enable task-oriented dialogue systems to steer the conversation towards successful task completion. In an end-to-end setting, a response can be constructed in a word-level sequential decision making process with the entire system vocabulary as action space. Policies trained in such a fashion do not require expert-defined action spaces, but they have to deal with large action spaces and long trajectories, making RL impractical. Using the latent space of a variational model as action space alleviates this problem. However, current approaches use an uninformed prior for training and optimize the latent distribution solely on the context. It is therefore unclear whether the latent representation truly encodes the characteristics of different actions. In this paper, we explore three ways of leveraging an auxiliary task to shape the latent variable distribution: via pre-training, to obtain an informed prior, and via multitask learning. We choose response auto-encoding as the auxiliary task, as this captures the generative factors of dialogue responses while requiring low computational cost and neither additional data nor labels. Our approach yields a more action-characterized latent representations which support end-to-end dialogue policy optimization and achieves state-of-the-art success rates. These results warrant a more wide-spread use of RL in end-to-end dialogue models.