# Technical report: supervised training of convolutional spiking neural networks with PyTorch

arXiv: https://arxiv.org/abs/1911.10124 · HTML: https://mlsanity.com/paper/1911.10124/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2019.

## Built on by (most cited first, 13 of 13)

- [GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks](https://mlsanity.com/paper/2210.13768/) (2022)
- [Direct Learning-Based Deep Spiking Neural Networks: A Review](https://mlsanity.com/paper/2305.19725/) (2023)
- [RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networks](https://mlsanity.com/paper/2308.06787/) (2023)
- [SpGesture: Source-Free Domain-adaptive sEMG-based Gesture Recognition with Jaccard Attentive Spiking Neural Network](https://mlsanity.com/paper/2405.14398/) (2024)
- [Reliable Decision from Multiple Subtasks through Threshold Optimization: Content Moderation in the Wild](https://mlsanity.com/paper/2208.07522/) (2022)
- [KLIF: An optimized spiking neuron unit for tuning surrogate gradient slope and membrane potential](https://mlsanity.com/paper/2302.09238/) (2023)
- [Fast threshold optimization for multi-label audio tagging using Surrogate gradient learning](https://mlsanity.com/paper/2103.00833/) (2021)
- [Advanced Efficient Strategy for Detection of Dark Objects Based on Spiking Network with Multi-Box Detection](https://mlsanity.com/paper/2310.06370/) (2023)
- [Encrypted Internet traffic classification using a supervised Spiking Neural Network](https://mlsanity.com/paper/2101.09818/) (2021)
- [Spiking neural networks trained via proxy](https://mlsanity.com/paper/2109.13208/) (2021)
- [Membrane Potential Distribution Adjustment and Parametric Surrogate Gradient in Spiking Neural Networks](https://mlsanity.com/paper/2304.13289/) (2023)
- [Deep activity propagation via weight initialization in spiking neural networks](https://mlsanity.com/paper/2410.00580/) (2024)
- [STAER: Temporal Aligned Rehearsal for Continual Spiking Neural Network](https://mlsanity.com/paper/2601.20870/) (2026)

## Abstract (arXiv)

Recently, it has been shown that spiking neural networks (SNNs) can be trained efficiently, in a supervised manner, using backpropagation through time. Indeed, the most commonly used spiking neuron model, the leaky integrate-and-fire neuron, obeys a differential equation which can be approximated using discrete time steps, leading to a recurrent relation for the potential. The firing threshold causes optimization issues, but they can be overcome using a surrogate gradient. Here, we extend previous approaches in two ways. Firstly, we show that the approach can be used to train convolutional layers. Convolutions can be done in space, time (which simulates conduction delays), or both. Secondly, we include fast horizontal connections `a la Den`eve: when a neuron N fires, we subtract to the potentials of all the neurons with the same receptive the dot product between their weight vectors and the one of neuron N. As Den`eve et al. showed, this is useful to represent a dynamic multidimensional analog signal in a population of spiking neurons. Here we demonstrate that, in addition, such connections also allow implementing a multidimensional send-on-delta coding scheme. We validate our approach on one speech classification benchmarks: the Google speech command dataset. We managed to reach nearly state-of-the-art accuracy (94%) while maintaining low firing rates (about 5Hz). Our code is based on PyTorch and is available in open source at http://github.com/romainzimmer/s2net