# CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling

arXiv: https://arxiv.org/abs/2604.14489 · HTML: https://mlsanity.com/paper/2604.14489/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 51 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Builds on (most cited first, 18 of 18)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model](https://mlsanity.com/paper/2405.17978/) (2024)
- [Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shifts](https://mlsanity.com/paper/2210.03797/) (2022)
- [Neural Topic Modeling with Continual Lifelong Learning](https://mlsanity.com/paper/2006.10909/) (2020)
- [Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network](https://mlsanity.com/paper/2107.02757/) (2021)
- [On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling](https://mlsanity.com/paper/2401.14113/) (2024)
- [How Many Topics? Stability Analysis for Topic Models](https://mlsanity.com/paper/1404.4606/) (2014)
- [Convolutional Cobweb: A Model of Incremental Learning from 2D Images](https://mlsanity.com/paper/2201.06740/) (2022)
- [Efficient Induction of Language Models Via Probabilistic Concept Formation](https://mlsanity.com/paper/2212.11937/) (2022)
- [Incremental Concept Formation over Visual Images Without Catastrophic Forgetting](https://mlsanity.com/paper/2402.16933/) (2024)
- [Self-supervised Topic Taxonomy Discovery in the Box Embedding Space](https://mlsanity.com/paper/2408.15050/) (2024)
- [Taxonomic Networks: A Representation for Neuro-Symbolic Pairing](https://mlsanity.com/paper/2505.24601/) (2025)
- [Analysis and tuning of hierarchical topic models based on Renyi entropy approach](https://mlsanity.com/paper/2101.07598/) (2021)
- [HyHTM: Hyperbolic Geometry based Hierarchical Topic Models](https://mlsanity.com/paper/2305.09258/) (2023)

## Abstract (arXiv)

Topic modeling seeks to uncover latent semantic structure in text corpora with minimal supervision. Neural approaches achieve strong performance but require extensive tuning and struggle with lifelong learning due to catastrophic forgetting and fixed capacity, while classical probabilistic models lack flexibility and adaptability to streaming data. We introduce CobwebTM, a low-parameter lifelong hierarchical topic model based on incremental probabilistic concept formation. By adapting the Cobweb algorithm to continuous document embeddings, CobwebTM constructs semantic hierarchies online, enabling unsupervised topic discovery, dynamic topic creation, and hierarchical organization without predefining the number of topics. Across diverse datasets, CobwebTM achieves strong topic coherence, stable topics over time, and high-quality hierarchies, demonstrating that incremental symbolic concept formation combined with pretrained representations is an efficient approach to topic modeling.