# Evaluating Dynamic Topic Models

arXiv: https://arxiv.org/abs/2309.08627 · HTML: https://mlsanity.com/paper/2309.08627/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 64 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 10 of 10)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!](https://mlsanity.com/paper/2004.14914/) (2020)
- [The Dynamic Embedded Topic Model](https://mlsanity.com/paper/1907.05545/) (2019)
- [A Call for Standardization and Validation of Text Style Transfer Evaluation](https://mlsanity.com/paper/2306.00539/) (2023)
- [Scalable Generalized Dynamic Topic Models](https://mlsanity.com/paper/1803.07868/) (2018)
- [Scaling up Dynamic Topic Models](https://mlsanity.com/paper/1602.06049/) (2016)
- [Learning to Play Text-based Adventure Games with Maximum Entropy Reinforcement Learning](https://mlsanity.com/paper/2302.10720/) (2023)

## Built on by (most cited first, 4 of 4)

- [On the Challenges and Opportunities in Generative AI](https://mlsanity.com/paper/2403.00025/) (2024)
- [Stick-Breaking Embedded Topic Model with Continuous Optimal Transport for Online Analysis of Document Streams](https://mlsanity.com/paper/2510.18786/) (2025)
- [BERTilda: Explainable Topic Lifecycle Tracking with Split/Merge Detection via Similarity-and-Flow Temporal Graphs](https://mlsanity.com/paper/2608.18101/) (2026)
- [Dynamic Topic Modeling for Cross-Corpus Temporal Analysis](https://mlsanity.com/paper/2608.23284/) (2026)

## Abstract (arXiv)

There is a lack of quantitative measures to evaluate the progression of topics through time in dynamic topic models (DTMs). Filling this gap, we propose a novel evaluation measure for DTMs that analyzes the changes in the quality of each topic over time. Additionally, we propose an extension combining topic quality with the model's temporal consistency. We demonstrate the utility of the proposed measure by applying it to synthetic data and data from existing DTMs. We also conducted a human evaluation, which indicates that the proposed measure correlates well with human judgment. Our findings may help in identifying changing topics, evaluating different DTMs, and guiding future research in this area.