# Improving Variational Encoder-Decoders in Dialogue Generation

arXiv: https://arxiv.org/abs/1802.02032 · HTML: https://mlsanity.com/paper/1802.02032/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2018.

## Built on by (most cited first, 13 of 13)

- [Topic-Driven and Knowledge-Aware Transformer for Dialogue Emotion Detection](https://mlsanity.com/paper/2106.01071/) (2021)
- [Controllable Semantic Parsing via Retrieval Augmentation](https://mlsanity.com/paper/2110.08458/) (2021)
- [Controllable and Diverse Text Generation in E-commerce](https://mlsanity.com/paper/2102.11497/) (2021)
- [Continual Learning with Dirichlet Generative-based Rehearsal](https://mlsanity.com/paper/2309.06917/) (2023)
- [DLVGen: A Dual Latent Variable Approach to Personalized Dialogue Generation](https://mlsanity.com/paper/2111.11363/) (2021)
- [Hierarchical Inductive Transfer for Continual Dialogue Learning](https://mlsanity.com/paper/2203.10484/) (2022)
- [Rumor Detection on Social Media with Reinforcement Learning-based Key Propagation Graph Generator](https://mlsanity.com/paper/2405.13094/) (2024)
- [Taming Continuous Posteriors for Latent Variational Dialogue Policies](https://mlsanity.com/paper/2205.07633/) (2022)
- [Partially Randomizing Transformer Weights for Dialogue Response Diversity](https://mlsanity.com/paper/2311.10943/) (2023)
- [Adversarial Conversational Shaping for Intelligent Agents](https://mlsanity.com/paper/2307.11785/) (2023)
- [Boosting Summarization with Normalizing Flows and Aggressive Training](https://mlsanity.com/paper/2311.00588/) (2023)
- [Multi-party Response Generation with Relation Disentanglement](https://mlsanity.com/paper/2403.10827/) (2024)
- [A Multi-stage Constrained Optimization Framework for Data-driven Problems](https://mlsanity.com/paper/2607.23480/) (2026)

## Abstract (arXiv)

Variational encoder-decoders (VEDs) have shown promising results in dialogue generation. However, the latent variable distributions are usually approximated by a much simpler model than the powerful RNN structure used for encoding and decoding, yielding the KL-vanishing problem and inconsistent training objective. In this paper, we separate the training step into two phases: The first phase learns to autoencode discrete texts into continuous embeddings, from which the second phase learns to generalize latent representations by reconstructing the encoded embedding. In this case, latent variables are sampled by transforming Gaussian noise through multi-layer perceptrons and are trained with a separate VED model, which has the potential of realizing a much more flexible distribution. We compare our model with current popular models and the experiment demonstrates substantial improvement in both metric-based and human evaluations.