# SpikingGamma: Surrogate-Gradient Free and Temporally Precise Online Training of Spiking Neural Networks with Smoothed Delays

arXiv: https://arxiv.org/abs/2602.01978 · HTML: https://mlsanity.com/paper/2602.01978/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 25 of 25)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](https://mlsanity.com/paper/1910.07407/) (2019)
- [Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratch](https://mlsanity.com/paper/2010.01729/) (2020)
- [Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings](https://mlsanity.com/paper/2306.17670/) (2023)
- [Online Training Through Time for Spiking Neural Networks](https://mlsanity.com/paper/2210.04195/) (2022)
- [Surrogate gradients for analog neuromorphic computing](https://mlsanity.com/paper/2006.07239/) (2020)
- [Accurate online training of dynamical spiking neural networks through Forward Propagation Through Time](https://mlsanity.com/paper/2112.11231/) (2021)
- [Online Spatio-Temporal Learning in Deep Neural Networks](https://mlsanity.com/paper/2007.12723/) (2020)
- [Direct Training High-Performance Deep Spiking Neural Networks: A Review of Theories and Methods](https://mlsanity.com/paper/2405.04289/) (2024)
- [Batch Normalized Recurrent Neural Networks](https://mlsanity.com/paper/1510.01378/) (2015)
- [Co-learning synaptic delays, weights and adaptation in spiking neural networks](https://mlsanity.com/paper/2311.16112/) (2023)
- [Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation](https://mlsanity.com/paper/2408.07517/) (2024)
- [ETLP: Event-based Three-factor Local Plasticity for online learning with neuromorphic hardware](https://mlsanity.com/paper/2301.08281/) (2023)
- [Efficient Event-based Delay Learning in Spiking Neural Networks](https://mlsanity.com/paper/2501.07331/) (2025)
- [Evolved Neuromorphic Control for High Speed Divergence-based Landings of MAVs](https://mlsanity.com/paper/2003.03118/) (2020)
- [Directly Training Temporal Spiking Neural Network with Sparse Surrogate Gradient](https://mlsanity.com/paper/2406.19645/) (2024)
- [Exploring the Limitations of Layer Synchronization in Spiking Neural Networks](https://mlsanity.com/paper/2408.05098/) (2024)
- [Neuromorphic Sequential Arena: A Benchmark for Neuromorphic Temporal Processing](https://mlsanity.com/paper/2505.22035/) (2025)
- [Beyond Rate Coding: Surrogate Gradients Enable Spike Timing Learning in Spiking Neural Networks](https://mlsanity.com/paper/2507.16043/) (2025)
- [Estimating Post-Synaptic Effects for Online Training of Feed-Forward SNNs](https://mlsanity.com/paper/2311.16151/) (2023)

## Abstract (arXiv)

Neuromorphic hardware implementations of Spiking Neural Networks (SNNs) promise energy-efficient, low-latency AI through sparse, event-driven computation. Yet, training SNNs under fine temporal discretization remains a major challenge, hindering both low-latency responsiveness and the mapping of software-trained SNNs to efficient hardware. In current approaches, spiking neurons are modeled as self-recurrent units, embedded into recurrent networks to maintain state over time, and trained with BPTT or RTRL variants based on surrogate gradients. These methods scale poorly with temporal resolution, while online approximations often exhibit instability for long sequences and tend to fail at capturing temporal patterns precisely. To address these limitations, we develop spiking neurons with internal recursive memory structures that we combine with sigma-delta spike-coding. We show that this SpikingGamma model supports direct error backpropagation without surrogate gradients, can learn fine temporal patterns with minimal spiking in an online manner, and scale feedforward SNNs to complex tasks and benchmarks with competitive accuracy, all while being insensitive to the temporal resolution of the model. Our approach offers both an alternative to current recurrent SNNs trained with surrogate gradients, and a direct route for mapping SNNs to neuromorphic hardware.