mlsanityData to 7 Oct 2026

Toward Efficient Deep Spiking Neuron Networks:A Survey On Compression

arXiv 2407.08744 ↗First posted 3 Jun 2024cs.NE cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

With the rapid development of deep learning, Deep Spiking Neural Networks (DSNNs) have emerged as promising due to their unique spike event processing and asynchronous computation. When deployed on neuromorphic chips, DSNNs offer significant power advantages over Deep Artificial Neural Networks (DANNs) and eliminate time and energy consuming multiplications due to the binary nature of spikes (0 or 1). Additionally, DSNNs excel in processing temporal information, making them potentially superior for handling temporal data compared to DANNs. However, their deep network structure and numerous parameters result in high computational costs and energy consumption, limiting real-life deployment. To enhance DSNNs efficiency, researchers have adapted methods from DANNs, such as pruning, quantization, and knowledge distillation, and developed specific techniques like reducing spike firing and pruning time steps. While previous surveys have covered DSNNs algorithms, hardware deployment, and general overviews, focused research on DSNNs compression and efficiency has been lacking. This survey addresses this gap by concentrating on efficient DSNNs and their compression methods. It begins with an exploration of DSNNs' biological background and computational units, highlighting differences from DANNs. It then delves into various compression methods, including pruning, quantization, knowledge distillation, and reducing spike firing, and concludes with suggestions for future research directions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 79 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

42 of 79 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  5. 2020Knowledge Distillation: A Surveycited by 1,578
  6. 2019Event-based Vision: A Surveycited by 1,010
  7. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  8. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  9. 2020Image Segmentation Using Deep Learning: A Surveycited by 659
  10. 2019Object Detection in 20 Years: A Surveycited by 528
  11. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  12. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
30 more
  1. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
  2. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  3. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 163
  4. 2020Liquid Time-constant Networkscited by 160
  5. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  6. 2019Enabling Spike-based Backpropagation for Training Deep Neural Network Architecturescited by 119
  7. 2017Comparative Study of CNN and RNN for Natural Language Processingcited by 81
  8. 2020Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratchcited by 76
  9. 2020T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Codingcited by 45
  10. 2021Pruning of Deep Spiking Neural Networks through Gradient Rewiringcited by 41
  11. 2023Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillationcited by 41
  12. 2022AutoSNN: Towards Energy-Efficient Spiking Neural Networkscited by 40
  13. 2023SEENN: Towards Temporal Spiking Early-Exit Neural Networkscited by 39
  14. 2019Comprehensive SNN Compression Using ADMM Optimization and Activity Regularizationcited by 38
  15. 2024SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networkscited by 38
  16. 2019Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?cited by 31
  17. 2020Distilling Spikes: Knowledge Distillation in Spiking Neural Networkscited by 24
  18. 2024Direct Training High-Performance Deep Spiking Neural Networks: A Review of Theories and Methodscited by 24
  19. 2023ESL-SNNs: An Evolutionary Structure Learning Strategy for Spiking Neural Networkscited by 22
  20. 2024Shrinking Your TimeStep: Towards Low-Latency Neuromorphic Object Recognition with Spiking Neural Networkcited by 19
  21. 2022SATA: Sparsity-Aware Training Accelerator for Spiking Neural Networkscited by 18
  22. 2021Spatio-Temporal Pruning and Quantization for Low-latency Spiking Neural Networkscited by 16
  23. 2018A Low Effort Approach to Structured CNN Design Using PCAcited by 14
  24. 2023A Unified Framework for Soft Threshold Pruningcited by 12
  25. 2022Adaptive Sparse Structure Development with Pruning and Regeneration for Spiking Neural Networkscited by 9
  26. 2023MINT: Multiplier-less INTeger Quantization for Energy Efficient Spiking Neural Networkscited by 8
  27. 2024LitE-SNN: Designing Lightweight and Efficient Spiking Neural Network through Spatial-Temporal Compressive Network Search and Joint Optimizationcited by 7
  28. 2023Deep Spiking Neural Networks with High Representation Similarity Model Visual Pathways of Macaque and Mousecited by 6
  29. 2023Are SNNs Truly Energy-efficient? - A Hardware Perspectivecited by 6
  30. 2023Temporal Conditioning Spiking Latent Variable Models of the Neural Response to Natural Visual Scenescited by 4

Built on by

Not cited yet.