# Theories of synaptic memory consolidation and intelligent plasticity for continual learning

arXiv: https://arxiv.org/abs/2405.16922 · HTML: https://mlsanity.com/paper/2405.16922/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 48 references that are in the map.
- First posted on arXiv 27 May 2024.

## Builds on (most cited first, 6 of 6)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization](https://mlsanity.com/paper/1802.01569/) (2018)
- [Continual Reinforcement Learning with Complex Synapses](https://mlsanity.com/paper/1802.07239/) (2018)
- [Synaptic metaplasticity in binarized neural networks](https://mlsanity.com/paper/2101.07592/) (2021)
- [Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments](https://mlsanity.com/paper/2201.00042/) (2021)

## Built on by (most cited first, 3 of 3)

- [Machine Unlearning using Forgetting Neural Networks](https://mlsanity.com/paper/2410.22374/) (2024)
- [SwitchCIT: Switching for Continual Instruction Tuning](https://mlsanity.com/paper/2407.11780/) (2024)
- [Continual Knowledge Updating in LLM Systems: Learning Through Multi-Timescale Memory Dynamics](https://mlsanity.com/paper/2605.05097/) (2026)

## Abstract (arXiv)

Humans and animals learn throughout life. Such continual learning is crucial for intelligence. In this chapter, we examine the pivotal role plasticity mechanisms with complex internal synaptic dynamics could play in enabling this ability in neural networks. By surveying theoretical research, we highlight two fundamental enablers for continual learning. First, synaptic plasticity mechanisms must maintain and evolve an internal state over several behaviorally relevant timescales. Second, plasticity algorithms must leverage the internal state to intelligently regulate plasticity at individual synapses to facilitate the seamless integration of new memories while avoiding detrimental interference with existing ones. Our chapter covers successful applications of these principles to deep neural networks and underscores the significance of synaptic metaplasticity in sustaining continual learning capabilities. Finally, we outline avenues for further research to understand the brain's superb continual learning abilities and harness similar mechanisms for artificial intelligence systems.