# Gradient Descent for Spiking Neural Networks

arXiv: https://arxiv.org/abs/1706.04698 · HTML: https://mlsanity.com/paper/1706.04698/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2017.

## Built on by (most cited first, 30 of 48)

- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation](https://mlsanity.com/paper/2205.00459/) (2022)
- [Reducing ANN-SNN Conversion Error through Residual Membrane Potential](https://mlsanity.com/paper/2302.02091/) (2023)
- [Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks](https://mlsanity.com/paper/2302.14311/) (2023)
- [Training Full Spike Neural Networks via Auxiliary Accumulation Pathway](https://mlsanity.com/paper/2301.11929/) (2023)
- [Real Spike: Learning Real-valued Spikes for Spiking Neural Networks](https://mlsanity.com/paper/2210.06686/) (2022)
- [High-performance deep spiking neural networks with 0.3 spikes per neuron](https://mlsanity.com/paper/2306.08744/) (2023)
- [Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence](https://mlsanity.com/paper/2106.01288/) (2021)
- [The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware](https://mlsanity.com/paper/2106.07030/) (2021)
- [One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latency](https://mlsanity.com/paper/2110.05929/) (2021)
- [Spiking Graph Neural Network on Riemannian Manifolds](https://mlsanity.com/paper/2410.17941/) (2024)
- [Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks](https://mlsanity.com/paper/2212.01232/) (2022)
- [Bayesian Continual Learning via Spiking Neural Networks](https://mlsanity.com/paper/2208.13723/) (2022)
- [An Exact Mapping From ReLU Networks to Spiking Neural Networks](https://mlsanity.com/paper/2212.12522/) (2022)
- [Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks](https://mlsanity.com/paper/2111.05955/) (2021)
- [Spiking Convolutional Neural Networks for Text Classification](https://mlsanity.com/paper/2406.19230/) (2024)
- [SpikE: spike-based embeddings for multi-relational graph data](https://mlsanity.com/paper/2104.13398/) (2021)
- [Spiking Generative Adversarial Networks With a Neural Network Discriminator: Local Training, Bayesian Models, and Continual Meta-Learning](https://mlsanity.com/paper/2111.01750/) (2021)
- [Event-LSTM: An Unsupervised and Asynchronous Learning-based Representation for Event-based Data](https://mlsanity.com/paper/2105.04216/) (2021)
- [Spike time displacement based error backpropagation in convolutional spiking neural networks](https://mlsanity.com/paper/2108.13621/) (2021)
- [Persistent learning signals and working memory without continuous attractors](https://mlsanity.com/paper/2308.12585/) (2023)
- [A Differentiable Point Process with Its Application to Spiking Neural Networks](https://mlsanity.com/paper/2106.00901/) (2021)
- [Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework](https://mlsanity.com/paper/2110.00214/) (2021)
- [EXODUS: Stable and Efficient Training of Spiking Neural Networks](https://mlsanity.com/paper/2205.10242/) (2022)
- [SpikeCLIP: A Contrastive Language-Image Pretrained Spiking Neural Network](https://mlsanity.com/paper/2310.06488/) (2023)
- [PLEIADES: Building Temporal Kernels with Orthogonal Polynomials](https://mlsanity.com/paper/2405.12179/) (2024)
- [Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules](https://mlsanity.com/paper/2206.00823/) (2022)
- [Intelligence Processing Units Accelerate Neuromorphic Learning](https://mlsanity.com/paper/2211.10725/) (2022)
- [Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions](https://mlsanity.com/paper/2211.16113/) (2022)
- [Sparse-firing regularization methods for spiking neural networks with time-to-first spike coding](https://mlsanity.com/paper/2307.13007/) (2023)

## Abstract (arXiv)

Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by the lack of efficient supervised learning algorithm for spiking networks. Here, we present a gradient descent method for optimizing spiking network models by introducing a differentiable formulation of spiking networks and deriving the exact gradient calculation. For demonstration, we trained recurrent spiking networks on two dynamic tasks: one that requires optimizing fast (~millisecond) spike-based interactions for efficient encoding of information, and a delayed memory XOR task over extended duration (~second). The results show that our method indeed optimizes the spiking network dynamics on the time scale of individual spikes as well as behavioral time scales. In conclusion, our result offers a general purpose supervised learning algorithm for spiking neural networks, thus advancing further investigations on spike-based computation.