# Learning to learn online with neuromodulated synaptic plasticity in spiking neural networks

arXiv: https://arxiv.org/abs/2206.12520 · HTML: https://mlsanity.com/paper/2206.12520/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 98 references that are in the map.
- First posted on arXiv 25 Jun 2022.

## Builds on (most cited first, 30 of 36)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Building machines that adapt and compute like brains](https://mlsanity.com/paper/1711.04203/) (2017)
- [ProMP: Proximal Meta-Policy Search](https://mlsanity.com/paper/1810.06784/) (2018)
- [Evolved Policy Gradients](https://mlsanity.com/paper/1802.04821/) (2018)
- [AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence](https://mlsanity.com/paper/1905.10985/) (2019)
- [Learning to Continually Learn](https://mlsanity.com/paper/2002.09571/) (2020)
- [Neural Optimizer Search with Reinforcement Learning](https://mlsanity.com/paper/1709.07417/) (2017)
- [Weight Agnostic Neural Networks](https://mlsanity.com/paper/1906.04358/) (2019)
- [Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules](https://mlsanity.com/paper/1801.05219/) (2018)
- [Meta-Q-Learning](https://mlsanity.com/paper/1910.00125/) (2019)
- [Evolving Reinforcement Learning Algorithms](https://mlsanity.com/paper/2101.03958/) (2021)
- [The BrainScaleS-2 accelerated neuromorphic system with hybrid plasticity](https://mlsanity.com/paper/2201.11063/) (2022)
- [Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity](https://mlsanity.com/paper/2002.10585/) (2020)
- [A Modern Self-Referential Weight Matrix That Learns to Modify Itself](https://mlsanity.com/paper/2202.05780/) (2022)
- [Meta-Learning Update Rules for Unsupervised Representation Learning](https://mlsanity.com/paper/1804.00222/) (2018)
- [Low-Power Neuromorphic Hardware for Signal Processing Applications](https://mlsanity.com/paper/1901.03690/) (2019)
- [Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets](https://mlsanity.com/paper/1901.09049/) (2019)
- [Online Few-shot Gesture Learning on a Neuromorphic Processor](https://mlsanity.com/paper/2008.01151/) (2020)
- [Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity](https://mlsanity.com/paper/2111.01760/) (2021)
- [Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks](https://mlsanity.com/paper/1705.07241/) (2017)
- [Neuromorphic Hardware learns to learn](https://mlsanity.com/paper/1903.06493/) (2019)

## Built on by (most cited first, 2 of 2)

- [Synaptic motor adaptation: A three-factor learning rule for adaptive robotic control in spiking neural networks](https://mlsanity.com/paper/2306.01906/) (2023)
- [Biological connectomes as a representation for the architecture of artificial neural networks](https://mlsanity.com/paper/2209.14406/) (2022)

## Abstract (arXiv)

We propose that in order to harness our understanding of neuroscience toward machine learning, we must first have powerful tools for training brain-like models of learning. Although substantial progress has been made toward understanding the dynamics of learning in the brain, neuroscience-derived models of learning have yet to demonstrate the same performance capabilities as methods in deep learning such as gradient descent. Inspired by the successes of machine learning using gradient descent, we demonstrate that models of neuromodulated synaptic plasticity from neuroscience can be trained in Spiking Neural Networks (SNNs) with a framework of learning to learn through gradient descent to address challenging online learning problems. This framework opens a new path toward developing neuroscience inspired online learning algorithms.