# Membrane Potential Distribution Adjustment and Parametric Surrogate Gradient in Spiking Neural Networks

arXiv: https://arxiv.org/abs/2304.13289 · HTML: https://mlsanity.com/paper/2304.13289/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 27 of 27)

- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [Training Deep Spiking Neural Networks using Backpropagation](https://mlsanity.com/paper/1608.08782/) (2016)
- [Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting](https://mlsanity.com/paper/2202.11946/) (2022)
- [RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network](https://mlsanity.com/paper/2003.01811/) (2020)
- [Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation](https://mlsanity.com/paper/2005.01807/) (2020)
- [Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures](https://mlsanity.com/paper/1903.06379/) (2019)
- [Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing](https://mlsanity.com/paper/1603.08270/) (2016)
- [Spiking Deep Networks with LIF Neurons](https://mlsanity.com/paper/1510.08829/) (2015)
- [Pruning of Deep Spiking Neural Networks through Gradient Rewiring](https://mlsanity.com/paper/2105.04916/) (2021)
- [Sparse Spiking Gradient Descent](https://mlsanity.com/paper/2105.08810/) (2021)
- [Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware](https://mlsanity.com/paper/1601.04187/) (2016)
- [DART: Distribution Aware Retinal Transform for Event-based Cameras](https://mlsanity.com/paper/1710.10800/) (2017)
- [Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network](https://mlsanity.com/paper/2003.02944/) (2020)
- [Spike-Train Level Backpropagation for Training Deep Recurrent Spiking Neural Networks](https://mlsanity.com/paper/1908.06378/) (2019)
- [Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks](https://mlsanity.com/paper/1805.07866/) (2018)
- [Constructing Accurate and Efficient Deep Spiking Neural Networks with Double-threshold and Augmented Schemes](https://mlsanity.com/paper/2005.03231/) (2020)
- [SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes](https://mlsanity.com/paper/1906.00851/) (2019)
- [Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks](https://mlsanity.com/paper/2002.06199/) (2020)
- [Network Plasticity as Bayesian Inference](https://mlsanity.com/paper/1504.05143/) (2015)
- [Technical report: supervised training of convolutional spiking neural networks with PyTorch](https://mlsanity.com/paper/1911.10124/) (2019)
- [Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors](https://mlsanity.com/paper/2102.06408/) (2021)

## Abstract (arXiv)

As an emerging network model, spiking neural networks (SNNs) have aroused significant research attentions in recent years. However, the energy-efficient binary spikes do not augur well with gradient descent-based training approaches. Surrogate gradient (SG) strategy is investigated and applied to circumvent this issue and train SNNs from scratch. Due to the lack of well-recognized SG selection rule, most SGs are chosen intuitively. We propose the parametric surrogate gradient (PSG) method to iteratively update SG and eventually determine an optimal surrogate gradient parameter, which calibrates the shape of candidate SGs. In SNNs, neural potential distribution tends to deviate unpredictably due to quantization error. We evaluate such potential shift and propose methodology for potential distribution adjustment (PDA) to minimize the loss of undesired pre-activations. Experimental results demonstrate that the proposed methods can be readily integrated with backpropagation through time (BPTT) algorithm and help modulated SNNs to achieve state-of-the-art performance on both static and dynamic dataset with fewer timesteps.