# Meta-learning Spiking Neural Networks with Surrogate Gradient Descent

arXiv: https://arxiv.org/abs/2201.10777 · HTML: https://mlsanity.com/paper/2201.10777/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 70 references that are in the map.
- First posted on arXiv 26 Jan 2022.

## Builds on (most cited first, 28 of 28)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Building machines that adapt and compute like brains](https://mlsanity.com/paper/1711.04203/) (2017)
- [Graph-Based Object Classification for Neuromorphic Vision Sensing](https://mlsanity.com/paper/1908.06648/) (2019)
- [Differentiable plasticity: training plastic neural networks with backpropagation](https://mlsanity.com/paper/1804.02464/) (2018)
- [Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratch](https://mlsanity.com/paper/2010.01729/) (2020)
- [Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networks](https://mlsanity.com/paper/2105.11654/) (2021)
- [Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules](https://mlsanity.com/paper/1801.05219/) (2018)
- [Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm](https://mlsanity.com/paper/1710.11622/) (2017)
- [Neural Architecture Search for Spiking Neural Networks](https://mlsanity.com/paper/2201.10355/) (2022)
- [Neuromorphic Deep Learning Machines](https://mlsanity.com/paper/1612.05596/) (2016)
- [Brain-Inspired Learning on Neuromorphic Substrates](https://mlsanity.com/paper/2010.11931/) (2020)
- [Brain-inspired global-local learning incorporated with neuromorphic computing](https://mlsanity.com/paper/2006.03226/) (2020)
- [Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets](https://mlsanity.com/paper/1901.09049/) (2019)
- [Neuromorphic electronic circuits for building autonomous cognitive systems](https://mlsanity.com/paper/1403.6428/) (2014)
- [Online Few-shot Gesture Learning on a Neuromorphic Processor](https://mlsanity.com/paper/2008.01151/) (2020)
- [Learning to Learn with Feedback and Local Plasticity](https://mlsanity.com/paper/2006.09549/) (2020)
- [Fast On-Device Adaptation for Spiking Neural Networks via Online-Within-Online Meta-Learning](https://mlsanity.com/paper/2103.03901/) (2021)
- [Rounding Methods for Neural Networks with Low Resolution Synaptic Weights](https://mlsanity.com/paper/1504.05767/) (2015)

## Built on by (most cited first, 10 of 10)

- [Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN](https://mlsanity.com/paper/2305.19868/) (2023)
- [Autaptic Synaptic Circuit Enhances Spatio-temporal Predictive Learning of Spiking Neural Networks](https://mlsanity.com/paper/2406.00405/) (2024)
- [SNNAX -- Spiking Neural Networks in JAX](https://mlsanity.com/paper/2409.02842/) (2024)
- [Meta-Learning in Spiking Neural Networks with Reward-Modulated STDP](https://mlsanity.com/paper/2306.04410/) (2023)
- [Flexible and Efficient Surrogate Gradient Modeling with Forward Gradient Injection](https://mlsanity.com/paper/2406.00177/) (2024)
- [Synaptic metaplasticity with multi-level memristive devices](https://mlsanity.com/paper/2306.12142/) (2023)
- [Artificial-Spiking Hierarchical Networks for Vision-Language Representation Learning](https://mlsanity.com/paper/2308.09455/) (2023)
- [Emulating Brain-like Rapid Learning in Neuromorphic Edge Computing](https://mlsanity.com/paper/2408.15800/) (2024)
- [Self-cross Feature based Spiking Neural Networks for Efficient Few-shot Learning](https://mlsanity.com/paper/2505.07921/) (2025)
- [Large Language Models Inference Engines based on Spiking Neural Networks](https://mlsanity.com/paper/2510.00133/) (2025)

## Abstract (arXiv)

Adaptive "life-long" learning at the edge and during online task performance is an aspirational goal of AI research. Neuromorphic hardware implementing Spiking Neural Networks (SNNs) are particularly attractive in this regard, as their real-time, event-based, local computing paradigm makes them suitable for edge implementations and fast learning. However, the long and iterative learning that characterizes state-of-the-art SNN training is incompatible with the physical nature and real-time operation of neuromorphic hardware. Bi-level learning, such as meta-learning is increasingly used in deep learning to overcome these limitations. In this work, we demonstrate gradient-based meta-learning in SNNs using the surrogate gradient method that approximates the spiking threshold function for gradient estimations. Because surrogate gradients can be made twice differentiable, well-established, and effective second-order gradient meta-learning methods such as Model Agnostic Meta Learning (MAML) can be used. We show that SNNs meta-trained using MAML match or exceed the performance of conventional ANNs meta-trained with MAML on event-based meta-datasets. Furthermore, we demonstrate the specific advantages that accrue from meta-learning: fast learning without the requirement of high precision weights or gradients. Our results emphasize how meta-learning techniques can become instrumental for deploying neuromorphic learning technologies on real-world problems.