# Membrane Potential Batch Normalization for Spiking Neural Networks

arXiv: https://arxiv.org/abs/2308.08359 · HTML: https://mlsanity.com/paper/2308.08359/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 57 references that are in the map.
- First posted on arXiv 16 Aug 2023.

## Builds on (most cited first, 30 of 33)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting](https://mlsanity.com/paper/2202.11946/) (2022)
- [Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/1903.06530/) (2019)
- [RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network](https://mlsanity.com/paper/2003.01811/) (2020)
- [Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation](https://mlsanity.com/paper/2005.01807/) (2020)
- [Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures](https://mlsanity.com/paper/1903.06379/) (2019)
- [A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration](https://mlsanity.com/paper/2106.06984/) (2021)
- [Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks](https://mlsanity.com/paper/2002.10085/) (2020)
- [Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratch](https://mlsanity.com/paper/2010.01729/) (2020)
- [DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks](https://mlsanity.com/paper/2008.03658/) (2020)
- [GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks](https://mlsanity.com/paper/2210.13768/) (2022)
- [LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing](https://mlsanity.com/paper/2011.06176/) (2020)
- [Online Training Through Time for Spiking Neural Networks](https://mlsanity.com/paper/2210.04195/) (2022)
- [Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillation](https://mlsanity.com/paper/2304.05627/) (2023)
- [Reducing ANN-SNN Conversion Error through Residual Membrane Potential](https://mlsanity.com/paper/2302.02091/) (2023)
- [A Biologically Plausible Supervised Learning Method for Spiking Neural Networks Using the Symmetric STDP Rule](https://mlsanity.com/paper/1812.06574/) (2018)
- [Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization](https://mlsanity.com/paper/2305.02099/) (2023)
- [Differentiable Learning-to-Normalize via Switchable Normalization](https://mlsanity.com/paper/1806.10779/) (2018)
- [Direct Learning-Based Deep Spiking Neural Networks: A Review](https://mlsanity.com/paper/2305.19725/) (2023)
- [ESL-SNNs: An Evolutionary Structure Learning Strategy for Spiking Neural Networks](https://mlsanity.com/paper/2306.03693/) (2023)
- [Real Spike: Learning Real-valued Spikes for Spiking Neural Networks](https://mlsanity.com/paper/2210.06686/) (2022)

## Built on by (most cited first, 30 of 35)

- [Spiking PointNet: Spiking Neural Networks for Point Clouds](https://mlsanity.com/paper/2310.06232/) (2023)
- [SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms](https://mlsanity.com/paper/2406.03287/) (2024)
- [Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks](https://mlsanity.com/paper/2401.04486/) (2024)
- [Spiking Transformer:Introducing Accurate Addition-Only Spiking Self-Attention for Transformer](https://mlsanity.com/paper/2503.00226/) (2025)
- [LM-HT SNN: Enhancing the Performance of SNN to ANN Counterpart through Learnable Multi-hierarchical Threshold Model](https://mlsanity.com/paper/2402.00411/) (2024)
- [STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networks](https://mlsanity.com/paper/2503.02689/) (2025)
- [Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networks](https://mlsanity.com/paper/2503.03144/) (2025)
- [Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networks](https://mlsanity.com/paper/2406.07862/) (2024)
- [DA-LIF: Dual Adaptive Leaky Integrate-and-Fire Model for Deep Spiking Neural Networks](https://mlsanity.com/paper/2502.10422/) (2025)
- [Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directions](https://mlsanity.com/paper/2409.02111/) (2024)
- [TS-SNN: Temporal Shift Module for Spiking Neural Networks](https://mlsanity.com/paper/2505.04165/) (2025)
- [Exploring the Limitations of Layer Synchronization in Spiking Neural Networks](https://mlsanity.com/paper/2408.05098/) (2024)
- [ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognition](https://mlsanity.com/paper/2409.01564/) (2024)
- [SpikeBottleNet: Spike-Driven Feature Compression Architecture for Edge-Cloud Co-Inference](https://mlsanity.com/paper/2410.08673/) (2024)
- [ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks](https://mlsanity.com/paper/2506.07720/) (2025)
- [Spiking Neural Network Feature Discrimination Boosts Modality Fusion](https://mlsanity.com/paper/2502.10423/) (2025)
- [Head-Tail-Aware KL Divergence in Knowledge Distillation for Spiking Neural Networks](https://mlsanity.com/paper/2504.20445/) (2025)
- [Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks](https://mlsanity.com/paper/2505.05375/) (2025)
- [Efficient Training of Spiking Neural Networks by Spike-aware Data Pruning](https://mlsanity.com/paper/2510.04098/) (2025)
- [Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradients](https://mlsanity.com/paper/2605.27412/) (2026)
- [A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks](https://mlsanity.com/paper/2606.11236/) (2026)
- [NiSNN-A: Non-iterative Spiking Neural Networks with Attention with Application to Motor Imagery EEG Classification](https://mlsanity.com/paper/2312.05643/) (2023)
- [When Bio-Inspired Computing meets Deep Learning: Low-Latency, Accurate, & Energy-Efficient Spiking Neural Networks from Artificial Neural Networks](https://mlsanity.com/paper/2312.06900/) (2023)
- [A dynamic vision sensor object recognition model based on trainable event-driven convolution and spiking attention mechanism](https://mlsanity.com/paper/2409.12691/) (2024)
- [Rethinking SNN Online Training and Deployment: Gradient-Coherent Learning via Hybrid-Driven LIF Model](https://mlsanity.com/paper/2410.07547/) (2024)
- [DTA: Dual Temporal-channel-wise Attention for Spiking Neural Networks](https://mlsanity.com/paper/2503.10052/) (2025)
- [ILIF: Temporal Inhibitory Leaky Integrate-and-Fire Neuron for Overactivation in Spiking Neural Networks](https://mlsanity.com/paper/2505.10371/) (2025)
- [Integer Binary-Range Alignment Neuron for Spiking Neural Networks](https://mlsanity.com/paper/2506.05679/) (2025)
- [Optimization of Low-Latency Spiking Neural Networks Utilizing Historical Dynamics of Refractory Periods](https://mlsanity.com/paper/2507.02960/) (2025)
- [Discretized Quadratic Integrate-and-Fire Neuron Model for Deep Spiking Neural Networks](https://mlsanity.com/paper/2510.05168/) (2025)

## Abstract (arXiv)

As one of the energy-efficient alternatives of conventional neural networks (CNNs), spiking neural networks (SNNs) have gained more and more interest recently. To train the deep models, some effective batch normalization (BN) techniques are proposed in SNNs. All these BNs are suggested to be used after the convolution layer as usually doing in CNNs. However, the spiking neuron is much more complex with the spatio-temporal dynamics. The regulated data flow after the BN layer will be disturbed again by the membrane potential updating operation before the firing function, i.e., the nonlinear activation. Therefore, we advocate adding another BN layer before the firing function to normalize the membrane potential again, called MPBN. To eliminate the induced time cost of MPBN, we also propose a training-inference-decoupled re-parameterization technique to fold the trained MPBN into the firing threshold. With the re-parameterization technique, the MPBN will not introduce any extra time burden in the inference. Furthermore, the MPBN can also adopt the element-wised form, while these BNs after the convolution layer can only use the channel-wised form. Experimental results show that the proposed MPBN performs well on both popular non-spiking static and neuromorphic datasets. Our code is open-sourced at hrefhttps://github.com/yfguo91/MPBNMPBN.