# Continual Learning Through Synaptic Intelligence

arXiv: https://arxiv.org/abs/1703.04200 · HTML: https://mlsanity.com/paper/1703.04200/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Mar 2017.

## Built on by (most cited first, 14 of 14)

- [Probing Representation Forgetting in Supervised and Unsupervised Continual Learning](https://mlsanity.com/paper/2203.13381/) (2022)
- [Continual Learning with Weight Interpolation](https://mlsanity.com/paper/2404.04002/) (2024)
- [A Topology-aware Graph Coarsening Framework for Continual Graph Learning](https://mlsanity.com/paper/2401.03077/) (2024)
- [Explain to Not Forget: Defending Against Catastrophic Forgetting with XAI](https://mlsanity.com/paper/2205.01929/) (2022)
- [CFA: Constraint-based Finetuning Approach for Generalized Few-Shot Object Detection](https://mlsanity.com/paper/2204.05220/) (2022)
- [Artificial Neuronal Ensembles with Learned Context Dependent Gating](https://mlsanity.com/paper/2301.07187/) (2023)
- [KrADagrad: Kronecker Approximation-Domination Gradient Preconditioned Stochastic Optimization](https://mlsanity.com/paper/2305.19416/) (2023)
- [Learning to Remember, Learn, and Forget in Attention-Based Models](https://mlsanity.com/paper/2602.09075/) (2026)
- [On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling](https://mlsanity.com/paper/2206.14581/) (2022)
- [In-Context Learning can Perform Continual Learning Like Humans](https://mlsanity.com/paper/2509.22764/) (2025)
- [Data-Incremental Continual Offline Reinforcement Learning](https://mlsanity.com/paper/2404.12639/) (2024)
- [Hybrid Learners Do Not Forget: A Brain-Inspired Neuro-Symbolic Approach to Continual Learning](https://mlsanity.com/paper/2503.12635/) (2025)
- [ProgAgent:A Continual RL Agent with Progress-Aware Rewards](https://mlsanity.com/paper/2603.07784/) (2026)
- [Flawed in Nature, Perfect through Evolution](https://mlsanity.com/paper/2609.00129/) (2026)

## Abstract (arXiv)

While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning. In stark contrast, biological neural networks continually adapt to changing domains, possibly by leveraging complex molecular machinery to solve many tasks simultaneously. In this study, we introduce intelligent synapses that bring some of this biological complexity into artificial neural networks. Each synapse accumulates task relevant information over time, and exploits this information to rapidly store new memories without forgetting old ones. We evaluate our approach on continual learning of classification tasks, and show that it dramatically reduces forgetting while maintaining computational efficiency.