# Training Deep Spiking Neural Networks

arXiv: https://arxiv.org/abs/2006.04436 · HTML: https://mlsanity.com/paper/2006.04436/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2020.

## Built on by (most cited first, 21 of 21)

- [SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence](https://mlsanity.com/paper/2310.16620/) (2023)
- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [Neural Architecture Search for Spiking Neural Networks](https://mlsanity.com/paper/2201.10355/) (2022)
- [Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies](https://mlsanity.com/paper/2304.12760/) (2023)
- [Exploring Lottery Ticket Hypothesis in Spiking Neural Networks](https://mlsanity.com/paper/2207.01382/) (2022)
- [HALSIE: Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalities](https://mlsanity.com/paper/2211.10754/) (2022)
- [Rethinking the role of normalization and residual blocks for spiking neural networks](https://mlsanity.com/paper/2203.01544/) (2022)
- [EXODUS: Stable and Efficient Training of Spiking Neural Networks](https://mlsanity.com/paper/2205.10242/) (2022)
- [Spiking GATs: Learning Graph Attentions via Spiking Neural Network](https://mlsanity.com/paper/2209.13539/) (2022)
- [Temporal Contrastive Learning for Spiking Neural Networks](https://mlsanity.com/paper/2305.13909/) (2023)
- [GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL](https://mlsanity.com/paper/2404.15597/) (2024)
- [Evaluating Spiking Neural Network On Neuromorphic Platform For Human Activity Recognition](https://mlsanity.com/paper/2308.00787/) (2023)
- [Event-based Shape from Polarization with Spiking Neural Networks](https://mlsanity.com/paper/2312.16071/) (2023)
- [Time-independent Spiking Neuron via Membrane Potential Estimation for Efficient Spiking Neural Networks](https://mlsanity.com/paper/2409.04978/) (2024)
- [Pedestrian Crossing Intent Classification From Event-Based Vision Using Convolutional Spiking Neural Networks With Temporal Augmentation](https://mlsanity.com/paper/2609.13328/) (2026)
- [Exploring Tradeoffs in Spiking Neural Networks](https://mlsanity.com/paper/2212.09500/) (2022)
- [SPENSER: Towards a NeuroEvolutionary Approach for Convolutional Spiking Neural Networks](https://mlsanity.com/paper/2305.10987/) (2023)
- [InfLoR-SNN: Reducing Information Loss for Spiking Neural Networks](https://mlsanity.com/paper/2307.04356/) (2023)
- [Deep activity propagation via weight initialization in spiking neural networks](https://mlsanity.com/paper/2410.00580/) (2024)
- [IKUN: Initialization to Keep snn training and generalization great with sUrrogate-stable variaNce](https://mlsanity.com/paper/2411.18250/) (2024)
- [Balancing Uncertainty and Diversity of Samples: Leveraging Diversity of Least, High Confidence Samples for Effective Active Learning](https://mlsanity.com/paper/2605.22169/) (2026)

## Abstract (arXiv)

Computation using brain-inspired spiking neural networks (SNNs) with neuromorphic hardware may offer orders of magnitude higher energy efficiency compared to the current analog neural networks (ANNs). Unfortunately, training SNNs with the same number of layers as state of the art ANNs remains a challenge. To our knowledge the only method which is successful in this regard is supervised training of ANN and then converting it to SNN. In this work we directly train deep SNNs using backpropagation with surrogate gradient and find that due to implicitly recurrent nature of feed forward SNN's the exploding or vanishing gradient problem severely hinders their training. We show that this problem can be solved by tuning the surrogate gradient function. We also propose using batch normalization from ANN literature on input currents of SNN neurons. Using these improvements we show that is is possible to train SNN with ResNet50 architecture on CIFAR100 and Imagenette object recognition datasets. The trained SNN falls behind in accuracy compared to analogous ANN but requires several orders of magnitude less inference time steps (as low as 10) to reach good accuracy compared to SNNs obtained by conversion from ANN which require on the order of 1000 time steps.