# Continual Learning with Self-Organizing Maps

arXiv: https://arxiv.org/abs/1904.09330 · HTML: https://mlsanity.com/paper/1904.09330/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2019.

## Built on by (most cited first, 10 of 10)

- [Position: Theory of Mind Benchmarks are Broken for Large Language Models](https://mlsanity.com/paper/2412.19726/) (2024)
- [Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models](https://mlsanity.com/paper/2508.01908/) (2025)
- [Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inference](https://mlsanity.com/paper/2412.14355/) (2024)
- [Dendritic Self-Organizing Maps for Continual Learning](https://mlsanity.com/paper/2110.13611/) (2021)
- [The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2512.22337/) (2025)
- [Neuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps](https://mlsanity.com/paper/2402.12465/) (2024)
- [SatSOM: Saturation Self-Organizing Maps for Continual Learning](https://mlsanity.com/paper/2506.10680/) (2025)
- [Position: Collusion Risks Among AI Reasoning Agents Justify Certification Requirements for Making Market Decisions](https://mlsanity.com/paper/2608.18078/) (2026)
- [Class Incremental Continual Learning with Self-Organizing Maps and Synthetic Replay](https://mlsanity.com/paper/2508.21240/) (2025)
- [Unsupervised Continual Learning with Growing Self-Organizing Maps and Synthetic Replay](https://mlsanity.com/paper/2608.27662/) (2026)

## Abstract (arXiv)

Despite remarkable successes achieved by modern neural networks in a wide range of applications, these networks perform best in domain-specific stationary environments where they are trained only once on large-scale controlled data repositories. When exposed to non-stationary learning environments, current neural networks tend to forget what they had previously learned, a phenomena known as catastrophic forgetting. Most previous approaches to this problem rely on memory replay buffers which store samples from previously learned tasks, and use them to regularize the learning on new ones. This approach suffers from the important disadvantage of not scaling well to real-life problems in which the memory requirements become enormous. We propose a memoryless method that combines standard supervised neural networks with self-organizing maps to solve the continual learning problem. The role of the self-organizing map is to adaptively cluster the inputs into appropriate task contexts - without explicit labels - and allocate network resources accordingly. Thus, it selectively routes the inputs in accord with previous experience, ensuring that past learning is maintained and does not interfere with current learning. Out method is intuitive, memoryless, and performs on par with current state-of-the-art approaches on standard benchmarks.