# DelGrad: Exact event-based gradients for training delays and weights on spiking neuromorphic hardware

arXiv: https://arxiv.org/abs/2404.19165 · HTML: https://mlsanity.com/paper/2404.19165/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 71 references that are in the map.
- First posted on arXiv 30 Apr 2024.

## Builds on (most cited first, 30 of 33)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Long short-term memory and learning-to-learn in networks of spiking neurons](https://mlsanity.com/paper/1803.09574/) (2018)
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](https://mlsanity.com/paper/1910.07407/) (2019)
- [Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks](https://mlsanity.com/paper/2103.12593/) (2021)
- [Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks](https://mlsanity.com/paper/2009.08378/) (2020)
- [Fast and energy-efficient neuromorphic deep learning with first-spike times](https://mlsanity.com/paper/1912.11443/) (2019)
- [Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings](https://mlsanity.com/paper/2306.17670/) (2023)
- [Memory and information processing in neuromorphic systems](https://mlsanity.com/paper/1506.03264/) (2015)
- [The BrainScaleS-2 accelerated neuromorphic system with hybrid plasticity](https://mlsanity.com/paper/2201.11063/) (2022)
- [Surrogate gradients for analog neuromorphic computing](https://mlsanity.com/paper/2006.07239/) (2020)
- [A Long Short-Term Memory for AI Applications in Spike-based Neuromorphic Hardware](https://mlsanity.com/paper/2107.03992/) (2021)
- [High-performance deep spiking neural networks with 0.3 spikes per neuron](https://mlsanity.com/paper/2306.08744/) (2023)
- [Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence](https://mlsanity.com/paper/2106.01288/) (2021)
- [DYNAP-SE2: a scalable multi-core dynamic neuromorphic asynchronous spiking neural network processor](https://mlsanity.com/paper/2310.00564/) (2023)
- [The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware](https://mlsanity.com/paper/2106.07030/) (2021)
- [DenRAM: Neuromorphic Dendritic Architecture with RRAM for Efficient Temporal Processing with Delays](https://mlsanity.com/paper/2312.08960/) (2023)
- [Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate](https://mlsanity.com/paper/1912.12980/) (2019)
- [Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks](https://mlsanity.com/paper/2212.01232/) (2022)
- [Elucidating the theoretical underpinnings of surrogate gradient learning in spiking neural networks](https://mlsanity.com/paper/2404.14964/) (2024)
- [The Yin-Yang dataset](https://mlsanity.com/paper/2102.08211/) (2021)
- [Smooth Exact Gradient Descent Learning in Spiking Neural Networks](https://mlsanity.com/paper/2309.14523/) (2023)
- [jaxsnn: Event-driven Gradient Estimation for Analog Neuromorphic Hardware](https://mlsanity.com/paper/2401.16841/) (2024)
- [An Accelerated Analog Neuromorphic Hardware System Emulating NMDA- and Calcium-Based Non-Linear Dendrites](https://mlsanity.com/paper/1703.07286/) (2017)
- [Extending BrainScaleS OS for BrainScaleS-2](https://mlsanity.com/paper/2003.13750/) (2020)

## Built on by (most cited first, 9 of 9)

- [The Role of Temporal Hierarchy in Spiking Neural Networks](https://mlsanity.com/paper/2407.18838/) (2024)
- [Causal pieces: analysing and improving spiking neural networks piece by piece](https://mlsanity.com/paper/2504.14015/) (2025)
- [Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks](https://mlsanity.com/paper/2603.13283/) (2026)
- [Spike-based alignment learning solves the weight transport problem](https://mlsanity.com/paper/2503.02642/) (2025)
- [Three factor delay learning rules for spiking neural networks](https://mlsanity.com/paper/2601.00668/) (2026)
- [Equivalence of approximation by networks of single- and multi-spike neurons](https://mlsanity.com/paper/2603.13478/) (2026)
- [mGRADE: Minimal Recurrent Gating Meets Delay Convolutions for Lightweight Sequence Modeling](https://mlsanity.com/paper/2507.01829/) (2025)
- [Delays in Spiking Neural Networks: A State Space Model Approach](https://mlsanity.com/paper/2512.01906/) (2025)
- [Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization](https://mlsanity.com/paper/2607.14672/) (2026)

## Abstract (arXiv)

Spiking neural networks (SNNs) inherently rely on the timing of signals for representing and processing information. Incorporating trainable transmission delays, alongside synaptic weights, is crucial for shaping these temporal dynamics. While recent methods have shown the benefits of training delays and weights in terms of accuracy and memory efficiency, they rely on discrete time, approximate gradients, and full access to internal variables like membrane potentials. This limits their precision, efficiency, and suitability for neuromorphic hardware due to increased memory requirements and I/O bandwidth demands. To address these challenges, we propose DelGrad, an analytical, event-based method to compute exact loss gradients for both synaptic weights and delays. The inclusion of delays in the training process emerges naturally within our proposed formalism, enriching the model's search space with a temporal dimension. Moreover, DelGrad, grounded purely in spike timing, eliminates the need to track additional variables such as membrane potentials. To showcase this key advantage, we demonstrate the functionality and benefits of DelGrad on the BrainScaleS-2 neuromorphic platform, by training SNNs in a chip-in-the-loop fashion. For the first time, we experimentally demonstrate the memory efficiency and accuracy benefits of adding delays to SNNs on noisy mixed-signal hardware. Additionally, these experiments also reveal the potential of delays for stabilizing networks against noise. DelGrad opens a new way for training SNNs with delays on neuromorphic hardware, which results in fewer required parameters, higher accuracy and ease of hardware training.