Bayesian Metaplasticity from Synaptic Uncertainty
Not on the map. Why?
Abstract (arXiv)
Catastrophic forgetting remains a challenge for neural networks, especially in lifelong learning scenarios. In this study, we introduce MEtaplasticity from Synaptic Uncertainty (MESU), inspired by metaplasticity and Bayesian inference principles. MESU harnesses synaptic uncertainty to retain information over time, with its update rule closely approximating the diagonal Newton's method for synaptic updates. Through continual learning experiments on permuted MNIST tasks, we demonstrate MESU's remarkable capability to maintain learning performance across 100 tasks without the need of explicit task boundaries.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 23 references that are in the map. First posted on arXiv 15 Dec 2023.
Builds on
10 of 23 references.
References · 10
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
- 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
- 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
- 2019Three scenarios for continual learningcited by 614
- 2019Gradient based sample selection for online continual learningcited by 571
- 2018Progress & Compress: A scalable framework for continual learningcited by 520
- 2017Variational Continual Learningcited by 339
- 2021Synaptic metaplasticity in binarized neural networkscited by 29
- 2023Synaptic metaplasticity with multi-level memristive devicescited by 1
Built on by
Not cited yet.