mlsanityData to 7 Oct 2026

Efficient Training of Spiking Neural Networks with Temporally-Truncated Local Backpropagation through Time

arXiv 2201.07210 ↗First posted 13 Dec 2021cs.NE cs.LG eess.SPcited by 3

Not on the map. Why?

Abstract (arXiv)

Directly training spiking neural networks (SNNs) has remained challenging due to complex neural dynamics and intrinsic non-differentiability in firing functions. The well-known backpropagation through time (BPTT) algorithm proposed to train SNNs suffers from large memory footprint and prohibits backward and update unlocking, making it impossible to exploit the potential of locally-supervised training methods. This work proposes an efficient and direct training algorithm for SNNs that integrates a locally-supervised training method with a temporally-truncated BPTT algorithm. The proposed algorithm explores both temporal and spatial locality in BPTT and contributes to significant reduction in computational cost including GPU memory utilization, main memory access and arithmetic operations. We thoroughly explore the design space concerning temporal truncation length and local training block size and benchmark their impact on classification accuracy of different networks running different types of tasks. The results reveal that temporal truncation has a negative effect on the accuracy of classifying frame-based datasets, but leads to improvement in accuracy on dynamic-vision-sensor (DVS) recorded datasets. In spite of resulting information loss, local training is capable of alleviating overfitting. The combined effect of temporal truncation and local training can lead to the slowdown of accuracy drop and even improvement in accuracy. In addition, training deep SNNs models such as AlexNet classifying CIFAR10-DVS dataset leads to 7.26% increase in accuracy, 89.94% reduction in GPU memory, 10.79% reduction in memory access, and 99.64% reduction in MAC operations compared to the standard end-to-end BPTT.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 81 references that are in the map. First posted on arXiv 13 Dec 2021.

Builds on

39 of 81 references.

References · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  5. 2018Object Detection with Deep Learning: A Reviewcited by 524
  6. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  7. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  8. 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 396
  9. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
  10. 2015Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccadescited by 308
  11. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  12. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 262
27 more
  1. 2018Deep Learning in Spiking Neural Networkscited by 258
  2. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 257
  3. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  4. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 222
  5. 2016Training Deep Spiking Neural Networks using Backpropagationcited by 203
  6. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 174
  7. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  8. 2019Training Neural Networks with Local Error Signalscited by 135
  9. 2019Enabling Spike-based Backpropagation for Training Deep Neural Network Architecturescited by 119
  10. 2019Decoupled Greedy Learning of CNNscited by 76
  11. 2021Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networkscited by 74
  12. 2021Revisiting Locally Supervised Learning: an Alternative to End-to-end Trainingcited by 71
  13. 2016Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computingcited by 68
  14. 2019S4NN: temporal backpropagation for spiking neural networks with one spike per neuroncited by 47
  15. 2017Applications of Deep Learning and Reinforcement Learning to Biological Datacited by 45
  16. 2017Deep supervised learning using local errorscited by 41
  17. 2020Comparing SNNs and RNNs on Neuromorphic Vision Datasets: Similarities and Differencescited by 39
  18. 2018Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networkscited by 30
  19. 2016STDP-based spiking deep convolutional neural networks for object recognitioncited by 24
  20. 2021ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networkscited by 24
  21. 2017BP-STDP: Approximating Backpropagation using Spike Timing Dependent Plasticitycited by 23
  22. 2019Adaptively Truncating Backpropagation Through Time to Control Gradient Biascited by 22
  23. 2021The SpiNNaker 2 Processing Element Architecture for Hybrid Digital Neuromorphic Computingcited by 12
  24. 2019Design Space Exploration of Hardware Spiking Neurons for Embedded Artificial Intelligencecited by 7
  25. 2021Semi-supervised learning combining backpropagation and STDP: STDP enhances learning by backpropagation with a small amount of labeled data in a spiking neural networkcited by 3
  26. 2021Training Energy-Efficient Deep Spiking Neural Networks with Time-to-First-Spike Codingcited by 3
  27. 2020VOR Adaptation on a Humanoid iCub Robot Using a Spiking Cerebellar Modelcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Learning from Dense Events: Towards Fast Spiking Neural Networks Training via Event Dataset Distillationcited by 2
  2. 2026The Neural Division of Labor: Biologically-Inspired Modular Architectures for Robust Neuromorphic Computingcited by 0
  3. 2026Toward On-Chip Training of Spiking Neural Networks for Dense Event-Based Visioncited by 0