mlsanityData to 14 Oct 2026

Training Full Spike Neural Networks via Auxiliary Accumulation Pathway

arXiv 2301.11929 ↗First posted 27 Jan 2023cs.NE cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Due to the binary spike signals making converting the traditional high-power multiply-accumulation (MAC) into a low-power accumulation (AC) available, the brain-inspired Spiking Neural Networks (SNNs) are gaining more and more attention. However, the binary spike propagation of the Full-Spike Neural Networks (FSNN) with limited time steps is prone to significant information loss. To improve performance, several state-of-the-art SNN models trained from scratch inevitably bring many non-spike operations. The non-spike operations cause additional computational consumption and may not be deployed on some neuromorphic hardware where only spike operation is allowed. To train a large-scale FSNN with high performance, this paper proposes a novel Dual-Stream Training (DST) method which adds a detachable Auxiliary Accumulation Pathway (AAP) to the full spiking residual networks. The accumulation in AAP could compensate for the information loss during the forward and backward of full spike propagation, and facilitate the training of the FSNN. In the test phase, the AAP could be removed and only the FSNN remained. This not only keeps the lower energy consumption but also makes our model easy to deploy. Moreover, for some cases where the non-spike operations are available, the APP could also be retained in test inference and improve feature discrimination by introducing a little non-spike consumption. Extensive experiments on ImageNet, DVS Gesture, and CIFAR10-DVS datasets demonstrate the effectiveness of DST.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 44 of its 64 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

44 of 64 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  9. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  10. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  11. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 626
  12. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 522
32 more
  1. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 406
  2. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
  3. 2021Deep Residual Learning in Spiking Neural Networkscited by 331
  4. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 278
  5. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 271
  6. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 266
  7. 2018Deep Learning in Spiking Neural Networkscited by 263
  8. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 245
  9. 2016Training Deep Spiking Neural Networks using Backpropagationcited by 205
  10. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 198
  11. 2019ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blockscited by 178
  12. 2021Adversarial Reciprocal Points Learning for Open Set Recognitioncited by 153
  13. 2020RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networkcited by 145
  14. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 144
  15. 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 139
  16. 2019Enabling Spike-based Backpropagation for Training Deep Neural Network Architecturescited by 123
  17. 2021A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibrationcited by 117
  18. 2020Learning Open Set Network with Discriminative Reciprocal Pointscited by 115
  19. 2020Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networkscited by 100
  20. 2022Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representationcited by 89
  21. 2016Supervised learning based on temporal coding in spiking neural networkscited by 76
  22. 2020DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networkscited by 71
  23. 2021Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domaincited by 71
  24. 2015Spiking Deep Networks with LIF Neuronscited by 63
  25. 2019S4NN: temporal backpropagation for spiking neural networks with one spike per neuroncited by 49
  26. 2022Online Training Through Time for Spiking Neural Networkscited by 49
  27. 2017Gradient Descent for Spiking Neural Networkscited by 48
  28. 2019Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performancecited by 46
  29. 2019Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagationcited by 45
  30. 2020Unifying Activation- and Timing-based Learning Rules for Spiking Neural Networkscited by 28
  31. 2020Convolutional Spiking Neural Networks for Spatio-Temporal Feature Extractioncited by 23
  32. 2018Spiking Deep Residual Networkcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 6; 2025: 11; 2026: 3.

26 papers.

Sort:

By area:

  1. 2023AutoAgents: A Framework for Automatic Agent Generationcited by 216
  2. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 169
  3. 2023Spike-driven Transformercited by 146
  4. 2024QKFormer: Hierarchical Spiking Transformer using Q-K Attentioncited by 49
  5. 2023Learning Long Sequences in Spiking Neural Networkscited by 19
  6. 2025UCF-Crime-DVS: A Novel Event-Based Dataset for Video Anomaly Detection with Spiking Neural Networkscited by 19
  7. 2025STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networkscited by 11
  8. 2023Spikingformer: A Key Foundation Model for Spiking Neural Networkscited by 10
  9. 2024Spiking CenterNet: A Distillation-boosted Spiking Neural Network for Object Detectioncited by 8
  10. 2025TS-SNN: Temporal Shift Module for Spiking Neural Networkscited by 5
  11. 2025Three-Factor Learning in Spiking Neural Networks: An Overview of Methods and Trends from a Machine Learning Perspectivecited by 3
  12. 2023SynA-ResNet: Spike-driven ResNet Achieved through OR Residual Connectioncited by 2
14 more
  1. 2024Pedestrian intention prediction in Adverse Weather Conditions with Spiking Neural Networks and Dynamic Vision Sensorscited by 2
  2. 2024Efficient Speech Command Recognition Leveraging Spiking Neural Network and Curriculum Learning-based Knowledge Distillationcited by 2
  3. 2025Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendlinesscited by 2
  4. 2025SpikCommander: A High-performance Spiking Transformer with Multi-view Learning for Efficient Speech Command Recognitioncited by 2
  5. 2024EG-SpikeFormer: Eye-Gaze Guided Transformer on Spiking Neural Networks for Medical Image Analysiscited by 1
  6. 2025Spike-TBR: a Noise Resilient Neuromorphic Event Representationcited by 1
  7. 2025Unsupervised Backdoor Detection and Mitigation for Spiking Neural Networkscited by 1
  8. 2024Fourier or Wavelet bases as counterpart self-attention in spikformer for efficient visual classificationcited by 0
  9. 2025Neuromorphic Optical Tracking and Imaging of Randomly Moving Targets through Strongly Scattering Mediacited by 0
  10. 2025Stabilizing Direct Training of Spiking Neural Networks: Membrane Potential Initialization and Threshold-robust Surrogate Gradientcited by 0
  11. 2025Attention via Synaptic Plasticity is All You Need: A Biologically Inspired Spiking Neuromorphic Transformercited by 0
  12. 2026From Lightweight CNNs to SpikeNets: Benchmarking Accuracy-Energy Tradeoffs with Pruned Spiking SqueezeNetcited by 0
  13. 2026SpikeTAD: Spiking Neural Networks for End-to-End Temporal Action Detectioncited by 0
  14. 2026MSGAT: Multi-Head Spiking Graph Attention with Similarity-Space Fusion for Image-Text Retrievalcited by 0