mlsanityData to 7 Oct 2026

Stabilizing Direct Training of Spiking Neural Networks: Membrane Potential Initialization and Threshold-robust Surrogate Gradient

arXiv 2511.08708 ↗First posted 11 Nov 2025cs.NE cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advancements in the direct training of Spiking Neural Networks (SNNs) have demonstrated high-quality outputs even at early timesteps, paving the way for novel energy-efficient AI paradigms. However, the inherent non-linearity and temporal dependencies in SNNs introduce persistent challenges, such as temporal covariate shift (TCS) and unstable gradient flow with learnable neuron thresholds. In this paper, we present two key innovations: MP-Init (Membrane Potential Initialization) and TrSG (Threshold-robust Surrogate Gradient). MP-Init addresses TCS by aligning the initial membrane potential with its stationary distribution, while TrSG stabilizes gradient flow with respect to threshold voltage during training. Extensive experiments validate our approach, achieving state-of-the-art accuracy on both static and dynamic image datasets. The code is available at: https://github.com/kookhh0827/SNN-MP-Init-TRSG

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 56 references that are in the map. First posted on arXiv 11 Nov 2025.

Builds on

36 of 56 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  3. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  4. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  5. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 340
  6. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
  7. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  8. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
  9. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 251
  10. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 222
  11. 2016Training Deep Spiking Neural Networks using Backpropagationcited by 203
  12. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 192
24 more
  1. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 163
  2. 2023Spike-driven Transformercited by 141
  3. 2020Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratchcited by 76
  4. 2023Deep Directly-Trained Spiking Neural Networks for Object Detectioncited by 76
  5. 2016Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computingcited by 68
  6. 2015Spiking Deep Networks with LIF Neuronscited by 63
  7. 2022GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 63
  8. 2024QKFormer: Hierarchical Spiking Transformer using Q-K Attentioncited by 46
  9. 2023Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependenciescited by 41
  10. 2020Surrogate gradients for analog neuromorphic computingcited by 38
  11. 2024SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networkscited by 38
  12. 2023Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networkscited by 37
  13. 2023Membrane Potential Batch Normalization for Spiking Neural Networkscited by 35
  14. 2016Theory and Tools for the Conversion of Analog to Spiking Convolutional Neural Networkscited by 31
  15. 2023Direct Learning-Based Deep Spiking Neural Networks: A Reviewcited by 31
  16. 2024Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticketcited by 28
  17. 2023RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networkscited by 27
  18. 2023Training Full Spike Neural Networks via Auxiliary Accumulation Pathwaycited by 26
  19. 2024Direct Training High-Performance Deep Spiking Neural Networks: A Review of Theories and Methodscited by 24
  20. 2024CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 21
  21. 2022SATA: Sparsity-Aware Training Accelerator for Spiking Neural Networkscited by 18
  22. 2025Rethinking Spiking Neural Networks from an Ensemble Learning Perspectivecited by 12
  23. 2025Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacementcited by 6
  24. 2017The mean-field equation of a leaky integrate-and-fire neural network: measure solutions and steady statescited by 1

Built on by

Not cited yet.