# Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization

arXiv: https://arxiv.org/abs/2607.14672 · HTML: https://mlsanity.com/paper/2607.14672/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 76 references that are in the map.
- First posted on arXiv 16 Jul 2026.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Supervised learning based on temporal coding in spiking neural networks](https://mlsanity.com/paper/1606.08165/) (2016)
- [Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks](https://mlsanity.com/paper/2009.08378/) (2020)
- [Fast and energy-efficient neuromorphic deep learning with first-spike times](https://mlsanity.com/paper/1912.11443/) (2019)
- [S4NN: temporal backpropagation for spiking neural networks with one spike per neuron](https://mlsanity.com/paper/1910.09495/) (2019)
- [Gradient Descent for Spiking Neural Networks](https://mlsanity.com/paper/1706.04698/) (2017)
- [Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performance](https://mlsanity.com/paper/1909.10837/) (2019)
- [T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Coding](https://mlsanity.com/paper/2003.11741/) (2020)
- [The BrainScaleS-2 accelerated neuromorphic system with hybrid plasticity](https://mlsanity.com/paper/2201.11063/) (2022)
- [Neuromorphic Deep Learning Machines](https://mlsanity.com/paper/1612.05596/) (2016)
- [Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks](https://mlsanity.com/paper/1805.07866/) (2018)
- [Automatic Differentiation of Programs with Discrete Randomness](https://mlsanity.com/paper/2210.08572/) (2022)
- [Unifying Activation- and Timing-based Learning Rules for Spiking Neural Networks](https://mlsanity.com/paper/2006.02642/) (2020)
- [High-performance deep spiking neural networks with 0.3 spikes per neuron](https://mlsanity.com/paper/2306.08744/) (2023)
- [DYNAP-SE2: a scalable multi-core dynamic neuromorphic asynchronous spiking neural network processor](https://mlsanity.com/paper/2310.00564/) (2023)
- [Elucidating the theoretical underpinnings of surrogate gradient learning in spiking neural networks](https://mlsanity.com/paper/2404.14964/) (2024)
- [Smooth Exact Gradient Descent Learning in Spiking Neural Networks](https://mlsanity.com/paper/2309.14523/) (2023)
- [DelGrad: Exact event-based gradients for training delays and weights on spiking neuromorphic hardware](https://mlsanity.com/paper/2404.19165/) (2024)
- [Exact Gradients for Stochastic Spiking Neural Networks Driven by Rough Signals](https://mlsanity.com/paper/2405.13587/) (2024)
- [A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design](https://mlsanity.com/paper/2001.05348/) (2020)
- [Hardware Implementation of Spiking Neural Networks Using Time-To-First-Spike Encoding](https://mlsanity.com/paper/2006.05033/) (2020)
- [Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions](https://mlsanity.com/paper/2211.16113/) (2022)
- [Rethinking Skip Connections in Spiking Neural Networks with Time-To-First-Spike Coding](https://mlsanity.com/paper/2312.00919/) (2023)

## Abstract (arXiv)

Continuous-time spiking neural networks (SNNs) provide an event-driven framework for temporal computation, computational neuroscience, and neuromorphic hardware. However, training deep continuous-time SNNs is severely constrained by the memory required for exact spike-time computation, which evaluates and retains candidate firing times over intervals determined by presynaptic spike ordering. Here we introduce a memory-efficient training framework based on differentiable spike-time discretization (DSTD) for leaky integrate-and-fire neurons with general membrane and synaptic time constants. DSTD maps irregular presynaptic spikes onto differentiable weighted events at fixed time points, replacing the input-dependent candidate dimension with M fixed time intervals while accurately approximating continuous-time membrane-potential dynamics. This reduces candidate-related activation memory from O(NoutNin) to O(NoutM) in the case of time-to-first-spike (TTFS) coding, where Nin and Nout denote the numbers of presynaptic and postsynaptic neurons, respectively. We further introduce synfire-chain-inspired temporal regularization that organizes layer-wise firing windows, mitigates dead-neuron failures, and enables pipeline-like processing. In dense LIF layers, DSTD reduced peak memory consumption by up to approximately 100-fold and training time by up to approximately 20-fold compared with exact spike-time computation. Together, these methods allowed us to train 9-layer convolutional SNNs on CIFAR-10 and 20-layer convolutional SNNs on Fashion-MNIST on a single GPU.