# Are Neural Topic Models Broken?

arXiv: https://arxiv.org/abs/2210.16162 · HTML: https://mlsanity.com/paper/2210.16162/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 135 references that are in the map.
- First posted on arXiv 28 Oct 2022.

## Builds on (most cited first, 18 of 18)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Cross-lingual Contextualized Topic Models with Zero-shot Learning](https://mlsanity.com/paper/2004.07737/) (2020)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Neural Models for Documents with Metadata](https://mlsanity.com/paper/1705.09296/) (2017)
- [Topic-Guided Variational Autoencoders for Text Generation](https://mlsanity.com/paper/1903.07137/) (2019)
- [Improving Neural Topic Models using Knowledge Distillation](https://mlsanity.com/paper/2010.02377/) (2020)
- [How Many Topics? Stability Analysis for Topic Models](https://mlsanity.com/paper/1404.4606/) (2014)
- [What is Wrong with Topic Modeling? (and How to Fix it Using Search-based Software Engineering)](https://mlsanity.com/paper/1608.08176/) (2016)
- [Context Reinforced Neural Topic Modeling over Short Texts](https://mlsanity.com/paper/2008.04545/) (2020)
- [Topically Driven Neural Language Model](https://mlsanity.com/paper/1704.08012/) (2017)
- [Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling](https://mlsanity.com/paper/2104.05064/) (2021)
- [Measuring LDA Topic Stability from Clusters of Replicated Runs](https://mlsanity.com/paper/1808.08098/) (2018)
- [A Topic Coverage Approach to Evaluation of Topic Models](https://mlsanity.com/paper/2012.06274/) (2020)
- [A Deep and Autoregressive Approach for Topic Modeling of Multimodal Data](https://mlsanity.com/paper/1409.3970/) (2014)

## Built on by (most cited first, 11 of 11)

- [Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild](https://mlsanity.com/paper/2407.11438/) (2024)
- [FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model](https://mlsanity.com/paper/2405.17978/) (2024)
- [Sparse Autoencoders for Hypothesis Generation](https://mlsanity.com/paper/2502.04382/) (2025)
- [Explaining Datasets in Words: Statistical Models with Natural Language Parameters](https://mlsanity.com/paper/2409.08466/) (2024)
- [Contextualized Topic Coherence Metrics](https://mlsanity.com/paper/2305.14587/) (2023)
- [Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders](https://mlsanity.com/paper/2507.23220/) (2025)
- [Interactive Topic Models with Optimal Transport](https://mlsanity.com/paper/2406.19928/) (2024)
- [Enhancing Topic Interpretability for Neural Topic Modeling through Topic-wise Contrastive Learning](https://mlsanity.com/paper/2412.17338/) (2024)
- [Objectifying the Subjective: Cognitive Biases in Topic Interpretations](https://mlsanity.com/paper/2507.19117/) (2025)
- [Improving Topic Modeling by Distilling Soft Labels from Language Models](https://mlsanity.com/paper/2602.17907/) (2026)
- [Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiency](https://mlsanity.com/paper/2609.31460/) (2026)

## Abstract (arXiv)

Recently, the relationship between automated and human evaluation of topic models has been called into question. Method developers have staked the efficacy of new topic model variants on automated measures, and their failure to approximate human preferences places these models on uncertain ground. Moreover, existing evaluation paradigms are often divorced from real-world use. Motivated by content analysis as a dominant real-world use case for topic modeling, we analyze two related aspects of topic models that affect their effectiveness and trustworthiness in practice for that purpose: the stability of their estimates and the extent to which the model's discovered categories align with human-determined categories in the data. We find that neural topic models fare worse in both respects compared to an established classical method. We take a step toward addressing both issues in tandem by demonstrating that a straightforward ensembling method can reliably outperform the members of the ensemble.