Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization
Not on the map. Why?
Abstract (arXiv)
Continuous-time spiking neural networks (SNNs) provide an event-driven framework for temporal computation, computational neuroscience, and neuromorphic hardware. However, training deep continuous-time SNNs is severely constrained by the memory required for exact spike-time computation, which evaluates and retains candidate firing times over intervals determined by presynaptic spike ordering. Here we introduce a memory-efficient training framework based on differentiable spike-time discretization (DSTD) for leaky integrate-and-fire neurons with general membrane and synaptic time constants. DSTD maps irregular presynaptic spikes onto differentiable weighted events at fixed time points, replacing the input-dependent candidate dimension with M fixed time intervals while accurately approximating continuous-time membrane-potential dynamics. This reduces candidate-related activation memory from O(NoutNin) to O(NoutM) in the case of time-to-first-spike (TTFS) coding, where Nin and Nout denote the numbers of presynaptic and postsynaptic neurons, respectively. We further introduce synfire-chain-inspired temporal regularization that organizes layer-wise firing windows, mitigates dead-neuron failures, and enables pipeline-like processing. In dense LIF layers, DSTD reduced peak memory consumption by up to approximately 100-fold and training time by up to approximately 20-fold compared with exact spike-time computation. Together, these methods allowed us to train 9-layer convolutional SNNs on CIFAR-10 and 20-layer convolutional SNNs on Fashion-MNIST on a single GPU.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 76 references that are in the map. First posted on arXiv 16 Jul 2026.
Builds on
35 of 76 references.
References · 35
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
- 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
- 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
- 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 178
- 2016Supervised learning based on temporal coding in spiking neural networkscited by 75
- 2020Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networkscited by 59
- 2019Fast and energy-efficient neuromorphic deep learning with first-spike timescited by 52
- 2017Gradient Descent for Spiking Neural Networkscited by 47
23 more
- 2019S4NN: temporal backpropagation for spiking neural networks with one spike per neuroncited by 47
- 2019Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performancecited by 45
- 2020T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Codingcited by 45
- 2022The BrainScaleS-2 accelerated neuromorphic system with hybrid plasticitycited by 41
- 2016Neuromorphic Deep Learning Machinescited by 38
- 2018Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networkscited by 30
- 2022Automatic Differentiation of Programs with Discrete Randomnesscited by 28
- 2020Unifying Activation- and Timing-based Learning Rules for Spiking Neural Networkscited by 27
- 2023High-performance deep spiking neural networks with 0.3 spikes per neuroncited by 22
- 2023DYNAP-SE2: a scalable multi-core dynamic neuromorphic asynchronous spiking neural network processorcited by 20
- 2024Elucidating the theoretical underpinnings of surrogate gradient learning in spiking neural networkscited by 11
- 2023Smooth Exact Gradient Descent Learning in Spiking Neural Networkscited by 10
- 2024DelGrad: Exact event-based gradients for training delays and weights on spiking neuromorphic hardwarecited by 9
- 2024Exact Gradients for Stochastic Spiking Neural Networks Driven by Rough Signalscited by 9
- 2020A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Designcited by 4
- 2020Hardware Implementation of Spiking Neural Networks Using Time-To-First-Spike Encodingcited by 3
- 2022Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictionscited by 3
- 2023Rethinking Skip Connections in Spiking Neural Networks with Time-To-First-Spike Codingcited by 3
- 2026Training event-based neural networks with exact gradients via Differentiable ODE Solving in JAXcited by 2
- 2024Harnessing Nonidealities in Analog In-Memory Computing Circuits: A Physical Modeling Approach for Neuromorphic Systemscited by 1
- 2025CMOS Implementation of Field Programmable Spiking Neural Network for Hardware Reservoir Computingcited by 1
- 2026Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networkscited by 1
- 2026Quadratic integrate-and-fire neurons exhibit less fragmented loss landscapes and outperform leaky integrate-and-fire neurons in spike-based gradient descentcited by 1
Built on by
Not cited yet.