# Topic-Guided Variational Autoencoders for Text Generation

arXiv: https://arxiv.org/abs/1903.07137 · HTML: https://mlsanity.com/paper/1903.07137/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2019.

## Built on by (most cited first, 30 of 44)

- [A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models](https://mlsanity.com/paper/2201.05337/) (2022)
- [Latent Diffusion Energy-Based Model for Interpretable Text Modeling](https://mlsanity.com/paper/2206.05895/) (2022)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference](https://mlsanity.com/paper/2310.11142/) (2023)
- [Controllable Mixed-Initiative Dialogue Generation through Prompting](https://mlsanity.com/paper/2305.04147/) (2023)
- [Attribute Alignment: Controlling Text Generation from Pre-trained Language Models](https://mlsanity.com/paper/2103.11070/) (2021)
- [Math Word Problem Generation with Mathematical Consistency and Problem Context Constraints](https://mlsanity.com/paper/2109.04546/) (2021)
- [Mixture of Soft Prompts for Controllable Data Generation](https://mlsanity.com/paper/2303.01580/) (2023)
- [Why is constrained neural language generation particularly challenging?](https://mlsanity.com/paper/2206.05395/) (2022)
- [Variational Topic Inference for Chest X-Ray Report Generation](https://mlsanity.com/paper/2107.07314/) (2021)
- [DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM](https://mlsanity.com/paper/2310.15296/) (2023)
- [Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classification](https://mlsanity.com/paper/2108.11556/) (2021)
- [LLM-based multi-agent poetry generation in non-cooperative environments](https://mlsanity.com/paper/2409.03659/) (2024)
- [Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance](https://mlsanity.com/paper/2106.08007/) (2021)
- [Controllable and Diverse Text Generation in E-commerce](https://mlsanity.com/paper/2102.11497/) (2021)
- [Learning to Plan for Language Modeling from Unlabeled Data](https://mlsanity.com/paper/2404.00614/) (2024)
- [Are Neural Topic Models Broken?](https://mlsanity.com/paper/2210.16162/) (2022)
- [Constrained Text Generation with Global Guidance -- Case Study on CommonGen](https://mlsanity.com/paper/2103.07170/) (2021)
- [AdaVAE: Exploring Adaptive GPT-2s in Variational Auto-Encoders for Language Modeling](https://mlsanity.com/paper/2205.05862/) (2022)
- [Distributional Learning of Variational AutoEncoder: Application to Synthetic Data Generation](https://mlsanity.com/paper/2302.11294/) (2023)
- [Distantly Supervised Relation Extraction with Sentence Reconstruction and Knowledge Base Priors](https://mlsanity.com/paper/2104.08225/) (2021)
- [Sentiment-oriented Transformer-based Variational Autoencoder Network for Live Video Commenting](https://mlsanity.com/paper/2404.12782/) (2024)
- [Learning Graphon Autoencoders for Generative Graph Modeling](https://mlsanity.com/paper/2105.14244/) (2021)
- [Plug-and-Play Recipe Generation with Content Planning](https://mlsanity.com/paper/2212.05093/) (2022)
- [Advancing Precise Outline-Conditioned Text Generation with Task Duality and Explicit Outline Control](https://mlsanity.com/paper/2305.14459/) (2023)
- [Diversity-Aware Coherence Loss for Improving Neural Topic Models](https://mlsanity.com/paper/2305.16199/) (2023)
- [EXoN: EXplainable encoder Network](https://mlsanity.com/paper/2105.10867/) (2021)
- [CCGG: A Deep Autoregressive Model for Class-Conditional Graph Generation](https://mlsanity.com/paper/2110.03800/) (2021)
- [PCAE: A Framework of Plug-in Conditional Auto-Encoder for Controllable Text Generation](https://mlsanity.com/paper/2210.03496/) (2022)
- [Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review](https://mlsanity.com/paper/2602.11173/) (2026)

## Abstract (arXiv)

We propose a topic-guided variational autoencoder (TGVAE) model for text generation. Distinct from existing variational autoencoder (VAE) based approaches, which assume a simple Gaussian prior for the latent code, our model specifies the prior as a Gaussian mixture model (GMM) parametrized by a neural topic module. Each mixture component corresponds to a latent topic, which provides guidance to generate sentences under the topic. The neural topic module and the VAE-based neural sequence module in our model are learned jointly. In particular, a sequence of invertible Householder transformations is applied to endow the approximate posterior of the latent code with high flexibility during model inference. Experimental results show that our TGVAE outperforms alternative approaches on both unconditional and conditional text generation, which can generate semantically-meaningful sentences with various topics.