# Neural Variational Inference for Text Processing

arXiv: https://arxiv.org/abs/1511.06038 · HTML: https://mlsanity.com/paper/1511.06038/

## Key facts

- Cited by 111 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2015.

## Built on by (most cited first, 30 of 111)

- [Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning](https://mlsanity.com/paper/2301.11916/) (2023)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Latent Diffusion Energy-Based Model for Interpretable Text Modeling](https://mlsanity.com/paper/2206.05895/) (2022)
- [Topic-Driven and Knowledge-Aware Transformer for Dialogue Emotion Detection](https://mlsanity.com/paper/2106.01071/) (2021)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Consistency Regularization for Variational Auto-Encoders](https://mlsanity.com/paper/2105.14859/) (2021)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [Transformer-based Conditional Variational Autoencoder for Controllable Story Generation](https://mlsanity.com/paper/2101.00828/) (2021)
- [Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning](https://mlsanity.com/paper/2211.12878/) (2022)
- [Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings](https://mlsanity.com/paper/2203.01570/) (2022)
- [Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics](https://mlsanity.com/paper/2204.09874/) (2022)
- [FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model](https://mlsanity.com/paper/2405.17978/) (2024)
- [Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations](https://mlsanity.com/paper/2202.04582/) (2022)
- [Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity](https://mlsanity.com/paper/2502.13063/) (2025)
- [InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling](https://mlsanity.com/paper/2304.03544/) (2023)
- [Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network](https://mlsanity.com/paper/2107.02757/) (2021)
- [A Survey on Neural Topic Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2401.15351/) (2024)
- [vONTSS: vMF based semi-supervised neural topic modeling with optimal transport](https://mlsanity.com/paper/2307.01226/) (2023)
- [DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM](https://mlsanity.com/paper/2310.15296/) (2023)
- [Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion](https://mlsanity.com/paper/2405.17957/) (2024)
- [Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classification](https://mlsanity.com/paper/2108.11556/) (2021)
- [Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation](https://mlsanity.com/paper/2109.14885/) (2021)
- [HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding](https://mlsanity.com/paper/2210.10625/) (2022)
- [Guiding Visual Question Generation](https://mlsanity.com/paper/2110.08226/) (2021)
- [Neural Topic Modeling of Psychotherapy Sessions](https://mlsanity.com/paper/2204.10189/) (2022)
- [Topic Modeling as Multi-Objective Contrastive Optimization](https://mlsanity.com/paper/2402.07577/) (2024)
- [Few-shot Learning for Cross-Target Stance Detection by Aggregating Multimodal Embeddings](https://mlsanity.com/paper/2301.04535/) (2023)
- [DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations](https://mlsanity.com/paper/2306.16770/) (2023)
- [GradMem: Learning to Write Context into Memory with Test-Time Gradient Descent](https://mlsanity.com/paper/2603.13875/) (2026)

## Abstract (arXiv)

Recent advances in neural variational inference have spawned a renaissance in deep latent variable models. In this paper we introduce a generic variational inference framework for generative and conditional models of text. While traditional variational methods derive an analytic approximation for the intractable distributions over latent variables, here we construct an inference network conditioned on the discrete text input to provide the variational distribution. We validate this framework on two very different text modelling applications, generative document modelling and supervised question answering. Our neural variational document model combines a continuous stochastic document representation with a bag-of-words generative model and achieves the lowest reported perplexities on two standard test corpora. The neural answer selection model employs a stochastic representation layer within an attention mechanism to extract the semantics between a question and answer pair. On two question answering benchmarks this model exceeds all previous published benchmarks.