mlsanityData to 7 Oct 2026

InfLoR-SNN: Reducing Information Loss for Spiking Neural Networks

arXiv 2307.04356 ↗First posted 10 Jul 2023cs.NE cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The Spiking Neural Network (SNN) has attracted more and more attention recently. It adopts binary spike signals to transmit information. Benefitting from the information passing paradigm of SNNs, the multiplications of activations and weights can be replaced by additions, which are more energy-efficient. However, its "Hard Reset" mechanism for the firing activity would ignore the difference among membrane potentials when the membrane potential is above the firing threshold, causing information loss. Meanwhile, quantifying the membrane potential to 0/1 spikes at the firing instants will inevitably introduce the quantization error thus bringing about information loss too. To address these problems, we propose to use the "Soft Reset" mechanism for the supervised training-based SNNs, which will drive the membrane potential to a dynamic reset potential according to its magnitude, and Membrane Potential Rectifier (MPR) to reduce the quantization error via redistributing the membrane potential to a range close to the spikes. Results show that the SNNs with the "Soft Reset" mechanism and MPR outperform their vanilla counterparts on both static and dynamic datasets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

33 of 51 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  5. 2016Simple Online and Realtime Trackingcited by 862
  6. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  7. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  8. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  9. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  10. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  11. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 340
  12. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
21 more
  1. 2018Model compression via distillation and quantizationcited by 287
  2. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  3. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  4. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  5. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 222
  6. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 192
  7. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 174
  8. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  9. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 142
  10. 2020RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networkcited by 141
  11. 2019Enabling Spike-based Backpropagation for Training Deep Neural Network Architecturescited by 119
  12. 2023Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networkscited by 116
  13. 2021A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibrationcited by 111
  14. 2020Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networkscited by 98
  15. 2020DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networkscited by 70
  16. 2020T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Codingcited by 45
  17. 2018Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networkscited by 34
  18. 2023Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorizationcited by 34
  19. 2020Exploring the Connection Between Binary and Spiking Neural Networkscited by 33
  20. 2023Direct Learning-Based Deep Spiking Neural Networks: A Reviewcited by 31
  21. 2020Training Deep Spiking Neural Networkscited by 21

Built on by

Not cited yet.