# TACOS: Task Agnostic Continual Learning in Spiking Neural Networks

arXiv: https://arxiv.org/abs/2409.00021 · HTML: https://mlsanity.com/paper/2409.00021/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 16 Aug 2024.

## Builds on (most cited first, 20 of 20)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Overcoming Catastrophic Forgetting by Incremental Moment Matching](https://mlsanity.com/paper/1703.08475/) (2017)
- [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018)
- [Uncertainty-based Continual Learning with Adaptive Regularization](https://mlsanity.com/paper/1905.11614/) (2019)
- [Task Agnostic Continual Learning Using Online Variational Bayes](https://mlsanity.com/paper/1803.10123/) (2018)
- [Neuromorphic Deep Learning Machines](https://mlsanity.com/paper/1612.05596/) (2016)
- [Synaptic metaplasticity in binarized neural networks](https://mlsanity.com/paper/2101.07592/) (2021)
- [ASP: Learning to Forget with Adaptive Synaptic Plasticity in Spiking Neural Networks](https://mlsanity.com/paper/1703.07655/) (2017)
- [Controlled Forgetting: Targeted Stimulation and Dopaminergic Plasticity Modulation for Unsupervised Lifelong Learning in Spiking Neural Networks](https://mlsanity.com/paper/1902.03187/) (2019)
- [Sparse Meta Networks for Sequential Adaptation and its Application to Adaptive Language Modelling](https://mlsanity.com/paper/2009.01803/) (2020)
- [On the role of neurogenesis in overcoming catastrophic forgetting](https://mlsanity.com/paper/1811.02113/) (2018)

## Built on by (most cited first, 1 of 1)

- [Investigating Plausibility of Biologically Inspired Bayesian Learning in ANNs](https://mlsanity.com/paper/2411.18788/) (2024)

## Abstract (arXiv)

Catastrophic interference, the loss of previously learned information when learning new information, remains a major challenge in machine learning. Since living organisms do not seem to suffer from this problem, researchers have taken inspiration from biology to improve memory retention in artificial intelligence systems. However, previous attempts to use bio-inspired mechanisms have typically resulted in systems that rely on task boundary information during training and/or explicit task identification during inference, information that is not available in real-world scenarios. Here, we show that neuro-inspired mechanisms such as synaptic consolidation and metaplasticity can mitigate catastrophic interference in a spiking neural network, using only synapse-local information, with no need for task awareness, and with a fixed memory size that does not need to be increased when training on new tasks. Our model, TACOS, combines neuromodulation with complex synaptic dynamics to enable new learning while protecting previous information. We evaluate TACOS on sequential image recognition tasks and demonstrate its effectiveness in reducing catastrophic interference. Our results show that TACOS outperforms existing regularization techniques in domain-incremental learning scenarios. We also report the results of an ablation study to elucidate the contribution of each neuro-inspired mechanism separately.