# Neuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps

arXiv: https://arxiv.org/abs/2402.12465 · HTML: https://mlsanity.com/paper/2402.12465/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 68 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 30 of 30)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [Task-Free Continual Learning](https://mlsanity.com/paper/1812.03596/) (2018)
- [A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning](https://mlsanity.com/paper/2205.13218/) (2022)
- [Memory Efficient Experience Replay for Streaming Learning](https://mlsanity.com/paper/1809.05922/) (2018)
- [Class-Incremental Continual Learning into the eXtended DER-verse](https://mlsanity.com/paper/2201.00766/) (2022)
- [PyCIL: A Python Toolbox for Class-Incremental Learning](https://mlsanity.com/paper/2112.12533/) (2021)
- [Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets](https://mlsanity.com/paper/1503.01007/) (2015)
- [EEC: Learning to Encode and Regenerate Images for Continual Learning](https://mlsanity.com/paper/2101.04904/) (2021)
- [DualNet: Continual Learning, Fast and Slow](https://mlsanity.com/paper/2110.00175/) (2021)
- [Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations](https://mlsanity.com/paper/1810.07411/) (2018)
- [SCALE: Online Self-Supervised Lifelong Learning without Prior Knowledge](https://mlsanity.com/paper/2208.11266/) (2022)
- [Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting](https://mlsanity.com/paper/1905.10696/) (2019)
- [Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment](https://mlsanity.com/paper/2312.09257/) (2023)
- [Continual Learning with Self-Organizing Maps](https://mlsanity.com/paper/1904.09330/) (2019)
- [The Neural State Pushdown Automata](https://mlsanity.com/paper/1909.05233/) (2019)
- [Continual Competitive Memory: A Neural System for Online Task-Free Lifelong Learning](https://mlsanity.com/paper/2106.13300/) (2021)
- [Improving Self-Organizing Maps with Unsupervised Feature Extraction](https://mlsanity.com/paper/2009.02174/) (2020)
- [Dendritic Self-Organizing Maps for Continual Learning](https://mlsanity.com/paper/2110.13611/) (2021)
- [Self-organizing maps and generalization: an algorithmic description of Numerosity and Variability Effects](https://mlsanity.com/paper/1802.09442/) (2018)
- [Minimally Supervised Learning using Topological Projections in Self-Organizing Maps](https://mlsanity.com/paper/2401.06923/) (2024)
- [Randomized Self Organizing Map](https://mlsanity.com/paper/2011.09534/) (2020)
- [Non-Euclidean Self-Organizing Maps](https://mlsanity.com/paper/2109.11769/) (2021)

## Built on by (most cited first, 1 of 1)

- [SatSOM: Saturation Self-Organizing Maps for Continual Learning](https://mlsanity.com/paper/2506.10680/) (2025)

## Abstract (arXiv)

An intelligent system capable of continual learning is one that can process and extract knowledge from potentially infinitely long streams of pattern vectors. The major challenge that makes crafting such a system difficult is known as catastrophic forgetting - an agent, such as one based on artificial neural networks (ANNs), struggles to retain previously acquired knowledge when learning from new samples. Furthermore, ensuring that knowledge is preserved for previous tasks becomes more challenging when input is not supplemented with task boundary information. Although forgetting in the context of ANNs has been studied extensively, there still exists far less work investigating it in terms of unsupervised architectures such as the venerable self-organizing map (SOM), a neural model often used in clustering and dimensionality reduction. While the internal mechanisms of SOMs could, in principle, yield sparse representations that improve memory retention, we observe that, when a fixed-size SOM processes continuous data streams, it experiences concept drift. In light of this, we propose a generalization of the SOM, the continual SOM (CSOM), which is capable of online unsupervised learning under a low memory budget. Our results, on benchmarks including MNIST, Kuzushiji-MNIST, and Fashion-MNIST, show almost a two times increase in accuracy, and CIFAR-10 demonstrates a state-of-the-art result when tested on (online) unsupervised class incremental learning setting.