# Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling

arXiv: https://arxiv.org/abs/2303.15973 · HTML: https://mlsanity.com/paper/2303.15973/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Builds on (most cited first, 12 of 12)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Efficient Object Localization Using Convolutional Networks](https://mlsanity.com/paper/1411.4280/) (2014)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Survey of Dropout Methods for Deep Neural Networks](https://mlsanity.com/paper/1904.13310/) (2019)
- [Effective and Efficient Dropout for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1904.03392/) (2019)
- [An empirical analysis of dropout in piecewise linear networks](https://mlsanity.com/paper/1312.6197/) (2013)
- [Towards Dropout Training for Convolutional Neural Networks](https://mlsanity.com/paper/1512.00242/) (2015)

## Built on by (most cited first, 3 of 3)

- [GINopic: Topic Modeling with Graph Isomorphism Network](https://mlsanity.com/paper/2404.02115/) (2024)
- [S2WTM: Spherical Sliced-Wasserstein Autoencoder for Topic Modeling](https://mlsanity.com/paper/2507.12451/) (2025)
- [Evaluating Negative Sampling Approaches for Neural Topic Models](https://mlsanity.com/paper/2503.18167/) (2025)

## Abstract (arXiv)

Dropout is a widely used regularization trick to resolve the overfitting issue in large feedforward neural networks trained on a small dataset, which performs poorly on the held-out test subset. Although the effectiveness of this regularization trick has been extensively studied for convolutional neural networks, there is a lack of analysis of it for unsupervised models and in particular, VAE-based neural topic models. In this paper, we have analyzed the consequences of dropout in the encoder as well as in the decoder of the VAE architecture in three widely used neural topic models, namely, contextualized topic model (CTM), ProdLDA, and embedded topic model (ETM) using four publicly available datasets. We characterize the dropout effect on these models in terms of the quality and predictive performance of the generated topics.