# On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generation

arXiv: https://arxiv.org/abs/1909.13668 · HTML: https://mlsanity.com/paper/1909.13668/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2019.

## Built on by (most cited first, 9 of 9)

- [Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approach](https://mlsanity.com/paper/2207.00128/) (2022)
- [Combining Variational Autoencoders and Physical Bias for Improved Microscopy Data Analysis](https://mlsanity.com/paper/2302.04216/) (2023)
- [Unsupervised Representation Disentanglement of Text: An Evaluation on Synthetic Datasets](https://mlsanity.com/paper/2106.03631/) (2021)
- [Exploring Latent Spaces of Tonal Music using Variational Autoencoders](https://mlsanity.com/paper/2311.03621/) (2023)
- [A Generative Framework for Predictive Modeling of Multiple Chronic Conditions Using Graph Variational Autoencoder and Bandit-Optimized Graph Neural Network](https://mlsanity.com/paper/2409.13671/) (2024)
- [CatVRNN: Generating Category Texts via Multi-task Learning](https://mlsanity.com/paper/2107.05219/) (2021)
- [The Neglected Sibling: Isotropic Gaussian Posterior for VAE](https://mlsanity.com/paper/2110.07383/) (2021)
- [Boosting Summarization with Normalizing Flows and Aggressive Training](https://mlsanity.com/paper/2311.00588/) (2023)
- [SEER-ZSL: Semantic Encoder-Enhanced Representations for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2312.13100/) (2023)

## Abstract (arXiv)

Variational Autoencoders (VAEs) are known to suffer from learning uninformative latent representation of the input due to issues such as approximated posterior collapse, or entanglement of the latent space. We impose an explicit constraint on the Kullback-Leibler (KL) divergence term inside the VAE objective function. While the explicit constraint naturally avoids posterior collapse, we use it to further understand the significance of the KL term in controlling the information transmitted through the VAE channel. Within this framework, we explore different properties of the estimated posterior distribution, and highlight the trade-off between the amount of information encoded in a latent code during training, and the generative capacity of the model.