# Continual Learning with Columnar Spiking Neural Networks

arXiv: https://arxiv.org/abs/2506.17169 · HTML: https://mlsanity.com/paper/2506.17169/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 19 of 19)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Towards Continual Reinforcement Learning: A Review and Perspectives](https://mlsanity.com/paper/2012.13490/) (2020)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [Understanding the Role of Training Regimes in Continual Learning](https://mlsanity.com/paper/2006.06958/) (2020)
- [Continual Lifelong Learning in Natural Language Processing: A Survey](https://mlsanity.com/paper/2012.09823/) (2020)
- [Continual Learning for Large Language Models: A Survey](https://mlsanity.com/paper/2402.01364/) (2024)
- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron](https://mlsanity.com/paper/1903.02440/) (2019)
- [Continuous learning of spiking networks trained with local rules](https://mlsanity.com/paper/2111.09553/) (2021)
- [CoLaNET -- A Spiking Neural Network with Columnar Layered Architecture for Classification](https://mlsanity.com/paper/2409.01230/) (2024)
- [Classifying Images with CoLaNET Spiking Neural Network -- the MNIST Example](https://mlsanity.com/paper/2409.07833/) (2024)
- [A Digital Machine Learning Algorithm Simulating Spiking Neural Network CoLaNET](https://mlsanity.com/paper/2503.17111/) (2025)

## Built on by (most cited first, 2 of 2)

- [Hybrid ANN-SNN Pipeline with Local Plasticity](https://mlsanity.com/paper/2606.20151/) (2026)
- [Spiking Neural Network Predicting Sequence of the External Worlds States in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2609.27459/) (2026)

## Abstract (arXiv)

Continual learning is a key feature of biological neural systems, but artificial neural networks often suffer from catastrophic forgetting. Instead of backpropagation, biologically plausible learning algorithms may enable stable continual learning. This study proposes columnar-organized spiking neural networks (SNNs) with local learning rules for continual learning and catastrophic forgetting. Using CoLaNET (Columnar Layered Network), we show that its microcolumns adapt most efficiently to new tasks when they lack shared structure with prior learning. We demonstrate how CoLaNET hyperparameters govern the trade-off between retaining old knowledge (stability) and acquiring new information (plasticity). We evaluate CoLaNET on two benchmarks: Permuted MNIST (ten sequential pixel-permuted tasks) and a two-task MNIST/EMNIST setup. Our model learns ten sequential tasks effectively, maintaining 92% accuracy on each. It shows low forgetting, with only 4% performance degradation on the first task after training on nine subsequent tasks.