# Neural Topic Modeling with Continual Lifelong Learning

arXiv: https://arxiv.org/abs/2006.10909 · HTML: https://mlsanity.com/paper/2006.10909/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2020.

## Built on by (most cited first, 24 of 24)

- [Continual Pre-training of Language Models](https://mlsanity.com/paper/2302.03241/) (2023)
- [Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning](https://mlsanity.com/paper/2112.02706/) (2021)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning](https://mlsanity.com/paper/2308.09303/) (2023)
- [How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition](https://mlsanity.com/paper/2207.07730/) (2022)
- [A Survey on Neural Topic Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2401.15351/) (2024)
- [Topic Modeling as Multi-Objective Contrastive Optimization](https://mlsanity.com/paper/2402.07577/) (2024)
- [Federated Continual Learning for Text Classification via Selective Inter-client Transfer](https://mlsanity.com/paper/2210.06101/) (2022)
- [ContCommRTD: A Distributed Content-based Misinformation-aware Community Detection System for Real-Time Disaster Reporting](https://mlsanity.com/paper/2301.12984/) (2023)
- [Towards Adversarially Robust Continual Learning](https://mlsanity.com/paper/2303.17764/) (2023)
- [CWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic Modeling](https://mlsanity.com/paper/2305.09329/) (2023)
- [In-context Continual Learning Assisted by an External Continual Learner](https://mlsanity.com/paper/2412.15563/) (2024)
- [A Neural Span-Based Continual Named Entity Recognition Model](https://mlsanity.com/paper/2302.12200/) (2023)
- [Few-shot Learning for Topic Modeling](https://mlsanity.com/paper/2104.09011/) (2021)
- [Multi-source Neural Topic Modeling in Multi-view Embedding Spaces](https://mlsanity.com/paper/2104.08551/) (2021)
- [Hierarchical Variational Auto-Encoding for Unsupervised Domain Generalization](https://mlsanity.com/paper/2101.09436/) (2021)
- [Text Classification Under Class Distribution Shift: A Survey](https://mlsanity.com/paper/2502.12965/) (2025)
- [Smaller Is Better: An Analysis of Instance Quantity/Quality Trade-off in Rehearsal-based Continual Learning](https://mlsanity.com/paper/2105.14106/) (2021)
- [Towards Generalising Neural Topical Representations](https://mlsanity.com/paper/2307.12564/) (2023)
- [PromptDSI: Prompt-based Rehearsal-free Continual Learning for Document Retrieval](https://mlsanity.com/paper/2406.12593/) (2024)
- [The Bayesian Approach to Continual Learning: An Overview](https://mlsanity.com/paper/2507.08922/) (2025)
- [Transport-Hub-Aware Spatial-Temporal Adaptive Graph Transformer for Traffic Flow Prediction](https://mlsanity.com/paper/2310.08328/) (2023)
- [Continual Neural Topic Model](https://mlsanity.com/paper/2508.15612/) (2025)
- [CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling](https://mlsanity.com/paper/2604.14489/) (2026)

## Abstract (arXiv)

Lifelong learning has recently attracted attention in building machine learning systems that continually accumulate and transfer knowledge to help future learning. Unsupervised topic modeling has been popularly used to discover topics from document collections. However, the application of topic modeling is challenging due to data sparsity, e.g., in a small collection of (short) documents and thus, generate incoherent topics and sub-optimal document representations. To address the problem, we propose a lifelong learning framework for neural topic modeling that can continuously process streams of document collections, accumulate topics and guide future topic modeling tasks by knowledge transfer from several sources to better deal with the sparse data. In the lifelong process, we particularly investigate jointly: (1) sharing generative homologies (latent topics) over lifetime to transfer prior knowledge, and (2) minimizing catastrophic forgetting to retain the past learning via novel selective data augmentation, co-training and topic regularization approaches. Given a stream of document collections, we apply the proposed Lifelong Neural Topic Modeling (LNTM) framework in modeling three sparse document collections as future tasks and demonstrate improved performance quantified by perplexity, topic coherence and information retrieval task.