# Scaling up Dynamic Topic Models

arXiv: https://arxiv.org/abs/1602.06049 · HTML: https://mlsanity.com/paper/1602.06049/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2016.

## Built on by (most cited first, 8 of 8)

- [Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion](https://mlsanity.com/paper/2405.17957/) (2024)
- [ANTM: An Aligned Neural Topic Model for Exploring Evolving Topics](https://mlsanity.com/paper/2302.01501/) (2023)
- [Evaluating Dynamic Topic Models](https://mlsanity.com/paper/2309.08627/) (2023)
- [Deep Probabilistic Graphical Modeling](https://mlsanity.com/paper/2104.12053/) (2021)
- [Recurrent Coupled Topic Modeling over Sequential Documents](https://mlsanity.com/paper/2106.13732/) (2021)
- [Scaling up Dynamic Edge Partition Models via Stochastic Gradient MCMC](https://mlsanity.com/paper/2403.00044/) (2024)
- [Comment-level Topic Drift Analysis in the Reddit Corpus](https://mlsanity.com/paper/2608.19133/) (2026)
- [Towards Automatic Evolution Tree Generation from Citation Graphs](https://mlsanity.com/paper/2609.09561/) (2026)

## Abstract (arXiv)

Dynamic topic models (DTMs) are very effective in discovering topics and capturing their evolution trends in time series data. To do posterior inference of DTMs, existing methods are all batch algorithms that scan the full dataset before each update of the model and make inexact variational approximations with mean-field assumptions. Due to a lack of a more scalable inference algorithm, despite the usefulness, DTMs have not captured large topic dynamics. This paper fills this research void, and presents a fast and parallelizable inference algorithm using Gibbs Sampling with Stochastic Gradient Langevin Dynamics that does not make any unwarranted assumptions. We also present a Metropolis-Hastings based O(1) sampler for topic assignments for each word token. In a distributed environment, our algorithm requires very little communication between workers during sampling (almost embarrassingly parallel) and scales up to large-scale applications. We are able to learn the largest Dynamic Topic Model to our knowledge, and learned the dynamics of 1,000 topics from 2.6 million documents in less than half an hour, and our empirical results show that our algorithm is not only orders of magnitude faster than the baselines but also achieves lower perplexity.