# The Dynamic Embedded Topic Model

arXiv: https://arxiv.org/abs/1907.05545 · HTML: https://mlsanity.com/paper/1907.05545/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2019.

## Built on by (most cited first, 30 of 41)

- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model](https://mlsanity.com/paper/2405.17978/) (2024)
- [Sequence-to-Sequence Learning with Latent Neural Grammars](https://mlsanity.com/paper/2109.01135/) (2021)
- [Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network](https://mlsanity.com/paper/2107.02757/) (2021)
- [Towards the TopMost: A Topic Modeling System Toolkit](https://mlsanity.com/paper/2309.06908/) (2023)
- [A Survey on Neural Topic Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2401.15351/) (2024)
- [Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion](https://mlsanity.com/paper/2405.17957/) (2024)
- [Event-Driven News Stream Clustering using Entity-Aware Contextual Embeddings](https://mlsanity.com/paper/2101.11059/) (2021)
- [The future is different: Large pre-trained language models fail in prediction tasks](https://mlsanity.com/paper/2211.00384/) (2022)
- [Neural Dynamic Focused Topic Model](https://mlsanity.com/paper/2301.10988/) (2023)
- [ANTM: An Aligned Neural Topic Model for Exploring Evolving Topics](https://mlsanity.com/paper/2302.01501/) (2023)
- [Evaluating Dynamic Topic Models](https://mlsanity.com/paper/2309.08627/) (2023)
- [CLI-RAG: A Retrieval-Augmented Framework for Clinically Structured and Context Aware Text Generation with LLMs](https://mlsanity.com/paper/2507.06715/) (2025)
- [A Joint Learning Approach for Semi-supervised Neural Topic Modeling](https://mlsanity.com/paper/2204.03208/) (2022)
- [Alternators For Sequence Modeling](https://mlsanity.com/paper/2405.11848/) (2024)
- [Research Topic Flows in Co-Authorship Networks](https://mlsanity.com/paper/2206.07980/) (2022)
- [What Does the Indian Parliament Discuss? An Exploratory Analysis of the Question Hour in the Lok Sabha](https://mlsanity.com/paper/2304.00235/) (2023)
- [Sig-Networks Toolkit: Signature Networks for Longitudinal Language Modelling](https://mlsanity.com/paper/2312.03523/) (2023)
- [Dynamic embedded topic models and change-point detection for exploring literary-historical hypotheses](https://mlsanity.com/paper/2401.13905/) (2024)
- [Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference](https://mlsanity.com/paper/2105.00819/) (2021)
- [Negative-Binomial Randomized Gamma Markov Processes for Heterogeneous Overdispersed Count Time Series](https://mlsanity.com/paper/2402.18995/) (2024)
- [tPARAFAC2: Tracking evolving patterns in (incomplete) temporal data](https://mlsanity.com/paper/2407.01356/) (2024)
- [Visualizing Temporal Topic Embeddings with a Compass](https://mlsanity.com/paper/2409.10649/) (2024)
- [From Noise to Signal: When Outliers Seed New Topics](https://mlsanity.com/paper/2603.18358/) (2026)
- [HistLens: Mapping Idea Change across Concepts and Corpora](https://mlsanity.com/paper/2604.11749/) (2026)
- [Automatic Monitoring Social Dynamics During Big Incidences: A Case Study of COVID-19 in Bangladesh](https://mlsanity.com/paper/2101.09667/) (2021)
- [Deep Probabilistic Graphical Modeling](https://mlsanity.com/paper/2104.12053/) (2021)
- [Jointly Dynamic Topic Model for Recognition of Lead-lag Relationship in Two Text Corpora](https://mlsanity.com/paper/2111.10846/) (2021)
- [Tracking Brand-Associated Polarity-Bearing Topics in User Reviews](https://mlsanity.com/paper/2301.07183/) (2023)
- [ATEM: A Topic Evolution Model for the Detection of Emerging Topics in Scientific Archives](https://mlsanity.com/paper/2306.02221/) (2023)

## Abstract (arXiv)

Topic modeling analyzes documents to learn meaningful patterns of words. For documents collected in sequence, dynamic topic models capture how these patterns vary over time. We develop the dynamic embedded topic model (D-ETM), a generative model of documents that combines dynamic latent Dirichlet allocation (D-LDA) and word embeddings. The D-ETM models each word with a categorical distribution parameterized by the inner product between the word embedding and a per-time-step embedding representation of its assigned topic. The D-ETM learns smooth topic trajectories by defining a random walk prior over the embedding representations of the topics. We fit the D-ETM using structured amortized variational inference with a recurrent neural network. On three different corpora---a collection of United Nations debates, a set of ACL abstracts, and a dataset of Science Magazine articles---we found that the D-ETM outperforms D-LDA on a document completion task. We further found that the D-ETM learns more diverse and coherent topics than D-LDA while requiring significantly less time to fit.