# Scalable Generalized Dynamic Topic Models

arXiv: https://arxiv.org/abs/1803.07868 · HTML: https://mlsanity.com/paper/1803.07868/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2018.

## Built on by (most cited first, 9 of 9)

- [Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion](https://mlsanity.com/paper/2405.17957/) (2024)
- [The future is different: Large pre-trained language models fail in prediction tasks](https://mlsanity.com/paper/2211.00384/) (2022)
- [Neural Dynamic Focused Topic Model](https://mlsanity.com/paper/2301.10988/) (2023)
- [ANTM: An Aligned Neural Topic Model for Exploring Evolving Topics](https://mlsanity.com/paper/2302.01501/) (2023)
- [Evaluating Dynamic Topic Models](https://mlsanity.com/paper/2309.08627/) (2023)
- [Recurrent Coupled Topic Modeling over Sequential Documents](https://mlsanity.com/paper/2106.13732/) (2021)
- [Expressive Mortality Models through Gaussian Process Kernels](https://mlsanity.com/paper/2305.01728/) (2023)
- [A Poisson-Gamma Dynamic Factor Model with Time-Varying Transition Dynamics](https://mlsanity.com/paper/2402.16297/) (2024)
- [Poisson-Gamma Dynamical Systems with Time-varying Transition Dynamics](https://mlsanity.com/paper/2609.00896/) (2026)

## Abstract (arXiv)

Dynamic topic models (DTMs) model the evolution of prevalent themes in literature, online media, and other forms of text over time. DTMs assume that word co-occurrence statistics change continuously and therefore impose continuous stochastic process priors on their model parameters. These dynamical priors make inference much harder than in regular topic models, and also limit scalability. In this paper, we present several new results around DTMs. First, we extend the class of tractable priors from Wiener processes to the generic class of Gaussian processes (GPs). This allows us to explore topics that develop smoothly over time, that have a long-term memory or are temporally concentrated (for event detection). Second, we show how to perform scalable approximate inference in these models based on ideas around stochastic variational inference and sparse Gaussian processes. This way we can train a rich family of DTMs to massive data. Our experiments on several large-scale datasets show that our generalized model allows us to find interesting patterns that were not accessible by previous approaches.