# A Stable Variational Autoencoder for Text Modelling

arXiv: https://arxiv.org/abs/1911.05343 · HTML: https://mlsanity.com/paper/1911.05343/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2019.

## Built on by (most cited first, 5 of 5)

- [Deep Extrapolation for Attribute-Enhanced Generation](https://mlsanity.com/paper/2107.02968/) (2021)
- [How do Variational Autoencoders Learn? Insights from Representational Similarity](https://mlsanity.com/paper/2205.08399/) (2022)
- [Affective Decoding for Empathetic Response Generation](https://mlsanity.com/paper/2108.08102/) (2021)
- [Coupled Variational Reinforcement Learning for Language Model General Reasoning](https://mlsanity.com/paper/2512.12576/) (2025)
- [Self-supervised Latent Space Optimization with Nebula Variational Coding](https://mlsanity.com/paper/2506.01414/) (2025)

## Abstract (arXiv)

Variational Autoencoder (VAE) is a powerful method for learning representations of high-dimensional data. However, VAEs can suffer from an issue known as latent variable collapse (or KL loss vanishing), where the posterior collapses to the prior and the model will ignore the latent codes in generative tasks. Such an issue is particularly prevalent when employing VAE-RNN architectures for text modelling (Bowman et al., 2016). In this paper, we present a simple architecture called holistic regularisation VAE (HR-VAE), which can effectively avoid latent variable collapse. Compared to existing VAE-RNN architectures, we show that our model can achieve much more stable training process and can generate text with significantly better quality.