# Continual Neural Topic Model

arXiv: https://arxiv.org/abs/2508.15612 · HTML: https://mlsanity.com/paper/2508.15612/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 59 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Reinforced Continual Learning](https://mlsanity.com/paper/1805.12369/) (2018)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Neural Variational Inference for Text Processing](https://mlsanity.com/paper/1511.06038/) (2015)
- [Cross-lingual Contextualized Topic Models with Zero-shot Learning](https://mlsanity.com/paper/2004.07737/) (2020)
- [The Dynamic Embedded Topic Model](https://mlsanity.com/paper/1907.05545/) (2019)
- [Neural Topic Modeling with Continual Lifelong Learning](https://mlsanity.com/paper/2006.10909/) (2020)
- [Enriching and Controlling Global Semantics for Text Summarization](https://mlsanity.com/paper/2109.10616/) (2021)
- [Capturing Greater Context for Question Generation](https://mlsanity.com/paper/1910.10274/) (2019)
- [Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion](https://mlsanity.com/paper/2405.17957/) (2024)
- [BERTrend: Neural Topic Modeling for Emerging Trends Detection](https://mlsanity.com/paper/2411.05930/) (2024)
- [TAN-NTM: Topic Attention Networks for Neural Topic Modeling](https://mlsanity.com/paper/2012.01524/) (2020)
- [ANTM: An Aligned Neural Topic Model for Exploring Evolving Topics](https://mlsanity.com/paper/2302.01501/) (2023)
- [How Pandemic Spread in News: Text Analysis Using Topic Model](https://mlsanity.com/paper/2102.04205/) (2021)

## Abstract (arXiv)

In continual learning, our aim is to learn a new task without forgetting what was learned previously. In topic models, this translates to learning new topic models without forgetting previously learned topics. Previous work either considered Dynamic Topic Models (DTMs), which learn the evolution of topics based on the entire training corpus at once, or Online Topic Models, which are updated continuously based on new data but do not have long-term memory. To fill this gap, we propose the Continual Neural Topic Model (CoNTM), which continuously learns topic models at subsequent time steps without forgetting what was previously learned. This is achieved using a global prior distribution that is continuously updated. In our experiments, CoNTM consistently outperformed the dynamic topic model in terms of topic quality and predictive perplexity while being able to capture topic changes online. The analysis reveals that CoNTM can learn more diverse topics and better capture temporal changes than existing methods.