# Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models

arXiv: https://arxiv.org/abs/1902.00154 · HTML: https://mlsanity.com/paper/1902.00154/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2019.

## Built on by (most cited first, 11 of 11)

- [The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation](https://mlsanity.com/paper/2109.06835/) (2021)
- [DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation](https://mlsanity.com/paper/2202.12350/) (2022)
- [PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation](https://mlsanity.com/paper/2203.09100/) (2022)
- [DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer](https://mlsanity.com/paper/2110.05999/) (2021)
- [Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis](https://mlsanity.com/paper/2206.09046/) (2022)
- [Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decoding](https://mlsanity.com/paper/2601.14304/) (2026)
- [Deep Learning Models in Software Requirements Engineering](https://mlsanity.com/paper/2105.07771/) (2021)
- [Plot Writing From Pre-Trained Language Models](https://mlsanity.com/paper/2206.03021/) (2022)
- [GCVAE: Generalized-Controllable Variational AutoEncoder](https://mlsanity.com/paper/2206.04225/) (2022)
- [POSGen: Personalized Opening Sentence Generation for Online Insurance Sales](https://mlsanity.com/paper/2302.06470/) (2023)
- [Writing Like the Best: Exemplar-Based Expository Text Generation](https://mlsanity.com/paper/2505.18859/) (2025)

## Abstract (arXiv)

Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables. In this paper, we investigate several multi-level structures to learn a VAE model to generate long, and coherent text. In particular, we use a hierarchy of stochastic layers between the encoder and decoder networks to generate more informative latent codes. We also investigate a multi-level decoder structure to learn a coherent long-term structure by generating intermediate sentence representations as high-level plan vectors. Empirical results demonstrate that a multi-level VAE model produces more coherent and less repetitive long text compared to the standard VAE models and can further mitigate the posterior-collapse issue.