# Synaptic metaplasticity in binarized neural networks

arXiv: https://arxiv.org/abs/2101.07592 · HTML: https://mlsanity.com/paper/2101.07592/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jan 2021.

## Built on by (most cited first, 29 of 29)

- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Bayesian Continual Learning via Spiking Neural Networks](https://mlsanity.com/paper/2208.13723/) (2022)
- [Deep learning via message passing algorithms based on belief propagation](https://mlsanity.com/paper/2110.14583/) (2021)
- [Statistical mechanics of continual learning: variational principle and mean-field potential](https://mlsanity.com/paper/2212.02846/) (2022)
- [Design Principles for Lifelong Learning AI Accelerators](https://mlsanity.com/paper/2310.04467/) (2023)
- [Bayesian continual learning and forgetting in neural networks](https://mlsanity.com/paper/2504.13569/) (2025)
- [The fine line between dead neurons and sparsity in binarized spiking neural networks](https://mlsanity.com/paper/2201.11915/) (2022)
- [Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks](https://mlsanity.com/paper/2402.11984/) (2024)
- [Bio-Inspired, Task-Free Continual Learning through Activity Regularization](https://mlsanity.com/paper/2212.04316/) (2022)
- [Eight challenges in developing theory of intelligence](https://mlsanity.com/paper/2306.11232/) (2023)
- [Random Quantum Neural Networks (RQNN) for Noisy Image Recognition](https://mlsanity.com/paper/2203.01764/) (2022)
- [Theories of synaptic memory consolidation and intelligent plasticity for continual learning](https://mlsanity.com/paper/2405.16922/) (2024)
- [On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling](https://mlsanity.com/paper/2206.14581/) (2022)
- [Synaptic metaplasticity with multi-level memristive devices](https://mlsanity.com/paper/2306.12142/) (2023)
- [TACOS: Task Agnostic Continual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2409.00021/) (2024)
- [Stochastic Engrams for Efficient Continual Learning](https://mlsanity.com/paper/2503.21436/) (2025)
- [Convergence and scaling of Boolean-weight optimization for hardware reservoirs](https://mlsanity.com/paper/2305.07908/) (2023)
- [Quantized Non-Volatile Nanomagnetic Synapse based Autoencoder for Efficient Unsupervised Network Anomaly Detection](https://mlsanity.com/paper/2309.06449/) (2023)
- [Bayesian Metaplasticity from Synaptic Uncertainty](https://mlsanity.com/paper/2312.10153/) (2023)
- [Quantified Task Misalignment to Inform PEFT: An Exploration of Domain Generalization and Catastrophic Forgetting in CLIP](https://mlsanity.com/paper/2402.09613/) (2024)
- [Neuromimetic metaplasticity for adaptive continual learning](https://mlsanity.com/paper/2407.07133/) (2024)
- [Investigating Plausibility of Biologically Inspired Bayesian Learning in ANNs](https://mlsanity.com/paper/2411.18788/) (2024)
- [Towards Experience Replay for Class-Incremental Learning in Fully-Binary Networks](https://mlsanity.com/paper/2503.07107/) (2025)
- [Generative Binary Memory: Pseudo-Replay Class-Incremental Learning on Binarized Embeddings](https://mlsanity.com/paper/2503.10333/) (2025)
- [What Neuroscience Can Teach AI About Learning in Continuously Changing Environments](https://mlsanity.com/paper/2507.02103/) (2025)
- [Active Continual Learning with Metaplastic Binary Bayesian Neural Networks](https://mlsanity.com/paper/2605.30198/) (2026)
- [NeuroGuard: Neural Gradient Update Aware of Representation Damage](https://mlsanity.com/paper/2608.08068/) (2026)
- [Metaplasticity as adaptive gradient preconditioning for incremental learning](https://mlsanity.com/paper/2608.14634/) (2026)
- [Do Artificial Neural Networks Need Dendritic Nanotubes? Two Reasons for Shrinking Importance Estimates in Continual Learning](https://mlsanity.com/paper/2610.14258/) (2026)

## Abstract (arXiv)

Unlike the brain, artificial neural networks, including state-of-the-art deep neural networks for computer vision, are subject to "catastrophic forgetting": they rapidly forget the previous task when trained on a new one. Neuroscience suggests that biological synapses avoid this issue through the process of synaptic consolidation and metaplasticity: the plasticity itself changes upon repeated synaptic events. In this work, we show that this concept of metaplasticity can be transferred to a particular type of deep neural networks, binarized neural networks, to reduce catastrophic forgetting.