BioGrad: Biologically Plausible Gradient-Based Learning for Spiking Neural Networks
Not on the map. Why?
Abstract (arXiv)
Spiking neural networks (SNN) are delivering energy-efficient, massively parallel, and low-latency solutions to AI problems, facilitated by the emerging neuromorphic chips. To harness these computational benefits, SNN need to be trained by learning algorithms that adhere to brain-inspired neuromorphic principles, namely event-based, local, and online computations. Yet, the state-of-the-art SNN training algorithms are based on backprop that does not follow the above principles. Due to its limited biological plausibility, the application of backprop to SNN requires non-local feedback pathways for transmitting continuous-valued errors, and relies on gradients from future timesteps. The introduction of biologically plausible modifications to backprop has helped overcome several of its limitations, but limits the degree to which backprop is approximated, which hinders its performance. We propose a biologically plausible gradient-based learning algorithm for SNN that is functionally equivalent to backprop, while adhering to all three neuromorphic principles. We introduced multi-compartment spiking neurons with local eligibility traces to compute the gradients required for learning, and a periodic "sleep" phase to further improve the approximation to backprop during which a local Hebbian rule aligns the feedback and feedforward weights. Our method achieved the same level of performance as backprop with multi-layer fully connected SNN on MNIST (98.13%) and the event-based N-MNIST (97.59%) datasets. We deployed our learning algorithm on Intel's Loihi to train a 1-hidden-layer network for MNIST, and obtained 93.32% test accuracy while consuming 400 times less energy per training sample than BioGrad on GPU. Our work shows that optimal learning is feasible in neuromorphic computing, and further pursuing its biological plausibility can better capture the benefits of this emerging computing paradigm.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 17 of its 45 references that are in the map. First posted on arXiv 27 Oct 2021.
Builds on
17 of 45 references.
References · 17
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2015Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccadescited by 314
- 2018SLAYER: Spike Layer Error Reassignment in Timecited by 266
- 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
- 2018Long short-term memory and learning-to-learn in networks of spiking neuronscited by 181
- 2015Adding Gradient Noise Improves Learning for Very Deep Networkscited by 145
- 2019Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challengescited by 93
- 2019Deep Learning without Weight Transportcited by 87
- 2016Towards deep learning with segregated dendritescited by 71
- 2016Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computingcited by 70
- 2020Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Controlcited by 49
- 2016Neuromorphic Deep Learning Machinescited by 38
5 more
- 2020Event-Driven Visual-Tactile Sensing and Learning for Robotscited by 29
- 2017BP-STDP: Approximating Backpropagation using Spike Timing Dependent Plasticitycited by 23
- 2019Spiking Neural Network on Neuromorphic Hardware for Energy-Efficient Unidimensional SLAMcited by 19
- 2019Rapid online learning and robust recall in a neuromorphic olfactory circuitcited by 18
- 2019Spike-based causal inference for weight alignmentcited by 10
Built on by
Citing papers per year: 2022: 2.
2 papers.