mlsanityData to 7 Oct 2026

Membrane Potential Batch Normalization for Spiking Neural Networks

arXiv 2308.08359 ↗First posted 16 Aug 2023cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

As one of the energy-efficient alternatives of conventional neural networks (CNNs), spiking neural networks (SNNs) have gained more and more interest recently. To train the deep models, some effective batch normalization (BN) techniques are proposed in SNNs. All these BNs are suggested to be used after the convolution layer as usually doing in CNNs. However, the spiking neuron is much more complex with the spatio-temporal dynamics. The regulated data flow after the BN layer will be disturbed again by the membrane potential updating operation before the firing function, i.e., the nonlinear activation. Therefore, we advocate adding another BN layer before the firing function to normalize the membrane potential again, called MPBN. To eliminate the induced time cost of MPBN, we also propose a training-inference-decoupled re-parameterization technique to fold the trained MPBN into the firing threshold. With the re-parameterization technique, the MPBN will not introduce any extra time burden in the inference. Furthermore, the MPBN can also adopt the element-wised form, while these BNs after the convolution layer can only use the channel-wised form. Experimental results show that the proposed MPBN performs well on both popular non-spiking static and neuromorphic datasets. Our code is open-sourced at hrefhttps://github.com/yfguo91/MPBNMPBN.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 33 of its 57 references that are in the map. First posted on arXiv 16 Aug 2023.

Builds on

33 of 57 references.

References · 33

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  3. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  4. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  5. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  6. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 340
  7. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
  8. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  9. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  10. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 222
  11. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 192
  12. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 174
21 more
  1. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 142
  2. 2020RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networkcited by 141
  3. 2019Enabling Spike-based Backpropagation for Training Deep Neural Network Architecturescited by 119
  4. 2021A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibrationcited by 111
  5. 2020Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networkscited by 98
  6. 2020Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratchcited by 76
  7. 2020DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networkscited by 70
  8. 2022GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 63
  9. 2020LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processingcited by 56
  10. 2022Online Training Through Time for Spiking Neural Networkscited by 48
  11. 2023Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillationcited by 41
  12. 2023Reducing ANN-SNN Conversion Error through Residual Membrane Potentialcited by 39
  13. 2018A Biologically Plausible Supervised Learning Method for Spiking Neural Networks Using the Symmetric STDP Rulecited by 36
  14. 2023Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorizationcited by 34
  15. 2018Differentiable Learning-to-Normalize via Switchable Normalizationcited by 33
  16. 2023Direct Learning-Based Deep Spiking Neural Networks: A Reviewcited by 31
  17. 2022Real Spike: Learning Real-valued Spikes for Spiking Neural Networkscited by 22
  18. 2023ESL-SNNs: An Evolutionary Structure Learning Strategy for Spiking Neural Networkscited by 22
  19. 2023Biologically inspired structure learning with reverse knowledge distillation for spiking neural networkscited by 11
  20. 2022Rethinking the role of normalization and residual blocks for spiking neural networkscited by 6
  21. 2023NeuroCLIP: Neuromorphic Data Understanding by CLIP and SNNcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 10; 2025: 16; 2026: 6.

35 papers.

Sort:

  1. 2023Spiking PointNet: Spiking Neural Networks for Point Cloudscited by 34
  2. 2024SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanismscited by 31
  3. 2024Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networkscited by 14
  4. 2025Spiking Transformer:Introducing Accurate Addition-Only Spiking Self-Attention for Transformercited by 13
  5. 2025STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networkscited by 11
  6. 2024LM-HT SNN: Enhancing the Performance of SNN to ANN Counterpart through Learnable Multi-hierarchical Threshold Modelcited by 10
  7. 2025Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networkscited by 10
  8. 2024Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networkscited by 9
  9. 2025DA-LIF: Dual Adaptive Leaky Integrate-and-Fire Model for Deep Spiking Neural Networkscited by 8
  10. 2024Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directionscited by 6
  11. 2025TS-SNN: Temporal Shift Module for Spiking Neural Networkscited by 5
  12. 2024Exploring the Limitations of Layer Synchronization in Spiking Neural Networkscited by 3
23 more
  1. 2024ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognitioncited by 3
  2. 2024SpikeBottleNet: Spike-Driven Feature Compression Architecture for Edge-Cloud Co-Inferencecited by 3
  3. 2025ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networkscited by 3
  4. 2025Spiking Neural Network Feature Discrimination Boosts Modality Fusioncited by 1
  5. 2025Head-Tail-Aware KL Divergence in Knowledge Distillation for Spiking Neural Networkscited by 1
  6. 2025Efficient Training of Spiking Neural Networks by Spike-aware Data Pruningcited by 1
  7. 2026Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradientscited by 1
  8. 2026A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networkscited by 1
  9. 2023NiSNN-A: Non-iterative Spiking Neural Networks with Attention with Application to Motor Imagery EEG Classificationcited by 0
  10. 2023When Bio-Inspired Computing meets Deep Learning: Low-Latency, Accurate, & Energy-Efficient Spiking Neural Networks from Artificial Neural Networkscited by 0
  11. 2024A dynamic vision sensor object recognition model based on trainable event-driven convolution and spiking attention mechanismcited by 0
  12. 2024Rethinking SNN Online Training and Deployment: Gradient-Coherent Learning via Hybrid-Driven LIF Modelcited by 0
  13. 2025DTA: Dual Temporal-channel-wise Attention for Spiking Neural Networkscited by 0
  14. 2025Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networkscited by 0
  15. 2025ILIF: Temporal Inhibitory Leaky Integrate-and-Fire Neuron for Overactivation in Spiking Neural Networkscited by 0
  16. 2025Integer Binary-Range Alignment Neuron for Spiking Neural Networkscited by 0
  17. 2025Optimization of Low-Latency Spiking Neural Networks Utilizing Historical Dynamics of Refractory Periodscited by 0
  18. 2025Discretized Quadratic Integrate-and-Fire Neuron Model for Deep Spiking Neural Networkscited by 0
  19. 2025Stabilizing Direct Training of Spiking Neural Networks: Membrane Potential Initialization and Threshold-robust Surrogate Gradientcited by 0
  20. 2026Learnable Burst Quantization for Expressive and Efficient Spiking Neural Networkscited by 0
  21. 2026NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Frameworkcited by 0
  22. 2026Intrinsically Stable Spiking Neural Networks: Overcoming the Performance Barrier in the Absence of Batch Normalizationcited by 0
  23. 2026Active Spiking Perception: The Membrane Potential as a Belief State for Anytime 3D Point Cloud Recognitioncited by 0