# Topic Modelling Meets Deep Neural Networks: A Survey

arXiv: https://arxiv.org/abs/2103.00498 · HTML: https://mlsanity.com/paper/2103.00498/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 85 references that are in the map.
- First posted on arXiv 28 Feb 2021.

## Builds on (most cited first, 30 of 45)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Neural Variational Inference for Text Processing](https://mlsanity.com/paper/1511.06038/) (2015)
- [Discovering Discrete Latent Topics with Neural Variational Inference](https://mlsanity.com/paper/1706.00359/) (2017)
- [Topic Modeling with Wasserstein Autoencoders](https://mlsanity.com/paper/1907.12374/) (2019)
- [Neural Topic Model via Optimal Transport](https://mlsanity.com/paper/2008.13537/) (2020)
- [Neural Models for Documents with Metadata](https://mlsanity.com/paper/1705.09296/) (2017)
- [Stick-Breaking Variational Autoencoders](https://mlsanity.com/paper/1605.06197/) (2016)
- [Topic-Guided Variational Autoencoders for Text Generation](https://mlsanity.com/paper/1903.07137/) (2019)
- [Dirichlet Variational Autoencoder](https://mlsanity.com/paper/1901.02739/) (2019)
- [Topic Modeling with Contextualized Word Representation Clusters](https://mlsanity.com/paper/2010.12626/) (2020)
- [WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling](https://mlsanity.com/paper/1803.01328/) (2018)
- [The Generalized Reparameterization Gradient](https://mlsanity.com/paper/1610.02287/) (2016)
- [Neural Topic Modeling with Bidirectional Adversarial Training](https://mlsanity.com/paper/2004.12331/) (2020)
- [Improving Topic Models with Latent Feature Word Representations](https://mlsanity.com/paper/1810.06306/) (2018)
- [TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency](https://mlsanity.com/paper/1611.01702/) (2016)
- [Coherence-Aware Neural Topic Modeling](https://mlsanity.com/paper/1809.02687/) (2018)
- [Topic Memory Networks for Short Text Classification](https://mlsanity.com/paper/1809.03664/) (2018)
- [Improving Neural Topic Models using Knowledge Distillation](https://mlsanity.com/paper/2010.02377/) (2020)
- [Neural Topic Modeling with Continual Lifelong Learning](https://mlsanity.com/paper/2006.10909/) (2020)
- [ATM:Adversarial-neural Topic Model](https://mlsanity.com/paper/1811.00265/) (2018)
- [Reparameterization Gradients through Acceptance-Rejection Sampling Algorithms](https://mlsanity.com/paper/1610.05683/) (2016)
- [Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks](https://mlsanity.com/paper/2010.06253/) (2020)
- [Gamma Belief Networks](https://mlsanity.com/paper/1512.03081/) (2015)
- [Neural Topic Modeling by Incorporating Document Relationship Graph](https://mlsanity.com/paper/2009.13972/) (2020)
- [Dirichlet belief networks for topic structure learning](https://mlsanity.com/paper/1811.00717/) (2018)
- [Neural Topic Modeling with Cycle-Consistent Adversarial Training](https://mlsanity.com/paper/2009.13971/) (2020)
- [Topic Compositional Neural Language Model](https://mlsanity.com/paper/1712.09783/) (2017)

## Built on by (most cited first, 30 of 51)

- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings](https://mlsanity.com/paper/2203.01570/) (2022)
- [Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics](https://mlsanity.com/paper/2204.09874/) (2022)
- [FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model](https://mlsanity.com/paper/2405.17978/) (2024)
- [Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network](https://mlsanity.com/paper/2107.02757/) (2021)
- [Towards the TopMost: A Topic Modeling System Toolkit](https://mlsanity.com/paper/2309.06908/) (2023)
- [Contextualized Topic Coherence Metrics](https://mlsanity.com/paper/2305.14587/) (2023)
- [A Survey on Neural Topic Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2401.15351/) (2024)
- [German Text Embedding Clustering Benchmark](https://mlsanity.com/paper/2401.02709/) (2024)
- [Are Neural Topic Models Broken?](https://mlsanity.com/paper/2210.16162/) (2022)
- [LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Models](https://mlsanity.com/paper/2406.09008/) (2024)
- [Challenges and Opportunities in Information Manipulation Detection: An Examination of Wartime Russian Media](https://mlsanity.com/paper/2205.12382/) (2022)
- [Neural Topic Modeling with Large Language Models in the Loop](https://mlsanity.com/paper/2411.08534/) (2024)
- [Automatic Histograms: Leveraging Language Models for Text Dataset Exploration](https://mlsanity.com/paper/2402.14880/) (2024)
- [Semantic-Driven Topic Modeling Using Transformer-Based Embeddings and Clustering Algorithms](https://mlsanity.com/paper/2410.00134/) (2024)
- [Neural Attention-Aware Hierarchical Topic Model](https://mlsanity.com/paper/2110.07161/) (2021)
- [Neural Multimodal Topic Modeling: A Comprehensive Evaluation](https://mlsanity.com/paper/2403.17308/) (2024)
- [Improving Contextualized Topic Models with Negative Sampling](https://mlsanity.com/paper/2303.14951/) (2023)
- [The Elements of Visual Art Recommendation: Learning Latent Semantic Representations of Paintings](https://mlsanity.com/paper/2303.08182/) (2023)
- [Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling](https://mlsanity.com/paper/2303.15973/) (2023)
- [Matching Visual Features to Hierarchical Semantic Topics for Image Paragraph Captioning](https://mlsanity.com/paper/2105.04143/) (2021)
- [No Pattern, No Recognition: a Survey about Reproducibility and Distortion Issues of Text Clustering and Topic Modeling](https://mlsanity.com/paper/2208.01712/) (2022)
- [PheME: A deep ensemble framework for improving phenotype prediction from multi-modal data](https://mlsanity.com/paper/2303.10794/) (2023)
- [Topic Modeling with Fine-tuning LLMs and Bag of Sentences](https://mlsanity.com/paper/2408.03099/) (2024)
- [Semantic-Driven Topic Modeling for Analyzing Creativity in Virtual Brainstorming](https://mlsanity.com/paper/2509.16835/) (2025)
- [Importance-aware Topic Modeling for Discovering Public Transit Risk from Noisy Social Media](https://mlsanity.com/paper/2512.06293/) (2025)
- [Adversarial Learned Fair Representations using Dampening and Stacking](https://mlsanity.com/paper/2203.08637/) (2022)
- [Topics in Contextualised Attention Embeddings](https://mlsanity.com/paper/2301.04339/) (2023)

## Abstract (arXiv)

Topic modelling has been a successful technique for text analysis for almost twenty years. When topic modelling met deep neural networks, there emerged a new and increasingly popular research area, neural topic models, with over a hundred models developed and a wide range of applications in neural language understanding such as text generation, summarisation and language models. There is a need to summarise research developments and discuss open problems and future directions. In this paper, we provide a focused yet comprehensive overview of neural topic models for interested researchers in the AI community, so as to facilitate them to navigate and innovate in this fast-growing research area. To the best of our knowledge, ours is the first review focusing on this specific topic.