# Beyond Rate Coding: Surrogate Gradients Enable Spike Timing Learning in Spiking Neural Networks

arXiv: https://arxiv.org/abs/2507.16043 · HTML: https://mlsanity.com/paper/2507.16043/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Builds on (most cited first, 14 of 14)

- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](https://mlsanity.com/paper/1910.07407/) (2019)
- [Efficient Neuromorphic Signal Processing with Loihi 2](https://mlsanity.com/paper/2111.03746/) (2021)
- [Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings](https://mlsanity.com/paper/2306.17670/) (2023)
- [Sparse Spiking Gradient Descent](https://mlsanity.com/paper/2105.08810/) (2021)
- [DenRAM: Neuromorphic Dendritic Architecture with RRAM for Efficient Temporal Processing with Delays](https://mlsanity.com/paper/2312.08960/) (2023)
- [Axonal Delay As a Short-Term Memory for Feed Forward Deep Spiking Neural Networks](https://mlsanity.com/paper/2205.02115/) (2022)
- [Adaptive Axonal Delays in feedforward spiking neural networks for accurate spoken word recognition](https://mlsanity.com/paper/2302.08607/) (2023)
- [Delayed Memory Unit: Modelling Temporal Dependency Through Delay Gate](https://mlsanity.com/paper/2310.14982/) (2023)
- [Adapting to time: Why nature may have evolved a diverse set of neurons](https://mlsanity.com/paper/2404.14325/) (2024)
- [Spikemax: Spike-based Loss Methods for Classification](https://mlsanity.com/paper/2205.09845/) (2022)

## Built on by (most cited first, 3 of 3)

- [SpikingGamma: Surrogate-Gradient Free and Temporally Precise Online Training of Spiking Neural Networks with Smoothed Delays](https://mlsanity.com/paper/2602.01978/) (2026)
- [Spark: Modular Spiking Neural Networks](https://mlsanity.com/paper/2602.02306/) (2026)
- [SpikeWorld: Fast-State Adaptation for Frozen Spiking World Models](https://mlsanity.com/paper/2608.07712/) (2026)

## Abstract (arXiv)

The surrogate gradient descent algorithm enabled spiking neural networks to be trained to carry out challenging sensory processing tasks, an important step in understanding how spikes contribute to neural computations. However, it is unclear the extent to which these algorithms fully explore the space of possible spiking solutions to problems. We investigated whether spiking networks trained with surrogate gradient descent can learn to make use of information that is only encoded in the timing and not the rate of spikes. We constructed synthetic datasets with a range of types of spike timing information (interspike intervals, spatio-temporal spike patterns or polychrony, and coincidence codes). We find that surrogate gradient descent training can extract all of these types of information. In more realistic speech-based datasets, both timing and rate information is present. We therefore constructed variants of these datasets in which all rate information is removed, and find that surrogate gradient descent can still perform well. We tested all networks both with and without trainable axonal delays. We find that delays can give a significant increase in performance, particularly for more challenging tasks. To determine what types of spike timing information are being used by the networks trained on the speech-based tasks, we test these networks on time-reversed spikes which perturb spatio-temporal spike patterns but leave interspike intervals and coincidence information unchanged. We find that when axonal delays are not used, networks perform well under time reversal, whereas networks trained with delays perform poorly. This suggests that spiking neural networks with delays are better able to exploit temporal structure. To facilitate further studies of temporal coding, we have released our modified speech-based datasets.