# Smooth Exact Gradient Descent Learning in Spiking Neural Networks

arXiv: https://arxiv.org/abs/2309.14523 · HTML: https://mlsanity.com/paper/2309.14523/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2023.

## Built on by (most cited first, 10 of 10)

- [DelGrad: Exact event-based gradients for training delays and weights on spiking neuromorphic hardware](https://mlsanity.com/paper/2404.19165/) (2024)
- [SparseProp: Efficient Event-Based Simulation and Training of Sparse Recurrent Spiking Neural Networks](https://mlsanity.com/paper/2312.17216/) (2023)
- [Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoders](https://mlsanity.com/paper/2404.04549/) (2024)
- [Causal pieces: analysing and improving spiking neural networks piece by piece](https://mlsanity.com/paper/2504.14015/) (2025)
- [Training event-based neural networks with exact gradients via Differentiable ODE Solving in JAX](https://mlsanity.com/paper/2603.08146/) (2026)
- [Quadratic integrate-and-fire neurons exhibit less fragmented loss landscapes and outperform leaky integrate-and-fire neurons in spike-based gradient descent](https://mlsanity.com/paper/2606.03935/) (2026)
- [A ghost mechanism: An analytical model of abrupt learning in recurrent networks](https://mlsanity.com/paper/2501.02378/) (2025)
- [Equivalence of approximation by networks of single- and multi-spike neurons](https://mlsanity.com/paper/2603.13478/) (2026)
- [Advanced Torrential Loss Function for Precipitation Forecasting](https://mlsanity.com/paper/2509.01348/) (2025)
- [Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization](https://mlsanity.com/paper/2607.14672/) (2026)

## Abstract (arXiv)

Gradient descent prevails in artificial neural network training, but seems inept for spiking neural networks as small parameter changes can cause sudden, disruptive (dis-)appearances of spikes. Here, we demonstrate exact gradient descent based on continuously changing spiking dynamics. These are generated by neuron models whose spikes vanish and appear at the end of a trial, where it cannot influence subsequent dynamics. This also enables gradient-based spike addition and removal. We illustrate our scheme with various tasks and setups, including recurrent and deep, initially silent networks.