# Unsupervised Continual Learning via Self-Adaptive Deep Clustering Approach

arXiv: https://arxiv.org/abs/2106.14563 · HTML: https://mlsanity.com/paper/2106.14563/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control](https://mlsanity.com/paper/1907.13322/) (2019)
- [Continual Learning via Inter-Task Synaptic Mapping](https://mlsanity.com/paper/2106.13954/) (2021)
- [Automatic Construction of Multi-layer Perceptron Network from Streaming Examples](https://mlsanity.com/paper/1910.03437/) (2019)

## Built on by (most cited first, 19 of 19)

- [The CLEAR Benchmark: Continual LEArning on Real-World Imagery](https://mlsanity.com/paper/2201.06289/) (2022)
- [Reinforced Continual Learning for Graphs](https://mlsanity.com/paper/2209.01556/) (2022)
- [Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects](https://mlsanity.com/paper/2312.17004/) (2023)
- [SCALE: Online Self-Supervised Lifelong Learning without Prior Knowledge](https://mlsanity.com/paper/2208.11266/) (2022)
- [Assessor-Guided Learning for Continual Environments](https://mlsanity.com/paper/2303.11624/) (2023)
- [Scalable Adversarial Online Continual Learning](https://mlsanity.com/paper/2209.01558/) (2022)
- [Few-Shot Continual Learning via Flat-to-Wide Approaches](https://mlsanity.com/paper/2306.14369/) (2023)
- [Few-Shot Class Incremental Learning via Robust Transformer Approach](https://mlsanity.com/paper/2405.05984/) (2024)
- [U-TELL: Unsupervised Task Expert Lifelong Learning](https://mlsanity.com/paper/2405.14623/) (2024)
- [Continual Knowledge Consolidation LORA for Domain Incremental Learning](https://mlsanity.com/paper/2510.16077/) (2025)
- [Cross-Domain Continual Learning via CLAMP](https://mlsanity.com/paper/2405.07142/) (2024)
- [Forward-Backward Knowledge Distillation for Continual Clustering](https://mlsanity.com/paper/2405.19234/) (2024)
- [Unsupervised Few-Shot Continual Learning for Remote Sensing Image Scene Classification](https://mlsanity.com/paper/2406.18574/) (2024)
- [Spatiotemporal Event Graphs for Dynamic Scene Understanding](https://mlsanity.com/paper/2312.07621/) (2023)
- [Sup3r: A Semi-Supervised Algorithm for increasing Sparsity, Stability, and Separability in Hierarchy Of Time-Surfaces architectures](https://mlsanity.com/paper/2404.12402/) (2024)
- [Caption, Create, Continue: Continual Learning with Pre-trained Generative Vision-Language Models](https://mlsanity.com/paper/2409.17806/) (2024)
- [Source-Free Cross-Domain Continual Learning](https://mlsanity.com/paper/2510.01649/) (2025)
- [Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidation](https://mlsanity.com/paper/2606.30190/) (2026)
- [Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models](https://mlsanity.com/paper/2607.09785/) (2026)

## Abstract (arXiv)

Unsupervised continual learning remains a relatively uncharted territory in the existing literature because the vast majority of existing works call for unlimited access of ground truth incurring expensive labelling cost. Another issue lies in the problem of task boundaries and task IDs which must be known for model's updates or model's predictions hindering feasibility for real-time deployment. Knowledge Retention in Self-Adaptive Deep Continual Learner, (KIERA), is proposed in this paper. KIERA is developed from the notion of flexible deep clustering approach possessing an elastic network structure to cope with changing environments in the timely manner. The centroid-based experience replay is put forward to overcome the catastrophic forgetting problem. KIERA does not exploit any labelled samples for model updates while featuring a task-agnostic merit. The advantage of KIERA has been numerically validated in popular continual learning problems where it shows highly competitive performance compared to state-of-the art approaches. Our implementation is available in textiturlhttps://github.com/ContinualAL/KIERA.