mlsanityData to 7 Oct 2026

Spike Accumulation Forwarding for Effective Training of Spiking Neural Networks

arXiv 2310.02772 ↗First posted 4 Oct 2023cs.NE cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

In this article, we propose a new paradigm for training spiking neural networks (SNNs), spike accumulation forwarding (SAF). It is known that SNNs are energy-efficient but difficult to train. Consequently, many researchers have proposed various methods to solve this problem, among which online training through time (OTTT) is a method that allows inferring at each time step while suppressing the memory cost. However, to compute efficiently on GPUs, OTTT requires operations with spike trains and weighted summation of spike trains during forwarding. In addition, OTTT has shown a relationship with the Spike Representation, an alternative training method, though theoretical agreement with Spike Representation has yet to be proven. Our proposed method can solve these problems; namely, SAF can halve the number of operations during the forward process, and it can be theoretically proven that SAF is consistent with the Spike Representation and OTTT, respectively. Furthermore, we confirmed the above contents through experiments and showed that it is possible to reduce memory and training time while maintaining accuracy.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 21 of its 44 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

21 of 44 references.

References · 21

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  3. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  4. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  5. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  6. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 257
  7. 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 178
  8. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 174
  9. 2020RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networkcited by 141
  10. 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 134
  11. 2022Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representationcited by 89
  12. 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 84
9 more
  1. 2021On Training Implicit Modelscited by 73
  2. 2022Online Training Through Time for Spiking Neural Networkscited by 48
  3. 2019Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performancecited by 45
  4. 2021Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium Statecited by 39
  5. 2021One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latencycited by 17
  6. 2019SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikescited by 15
  7. 2023Spiking Neural Network for Ultra-low-latency and High-accurate Object Detectioncited by 14
  8. 2015STDP as presynaptic activity times rate of change of postsynaptic activitycited by 8
  9. 2023SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networkscited by 8

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Magic for the Age of Quantized DNNscited by 0