mlsanityData to 14 Oct 2026

Exploring Lottery Ticket Hypothesis in Spiking Neural Networks

arXiv 2207.01382 ↗First posted 4 Jul 2022cs.AI cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

Spiking Neural Networks (SNNs) have recently emerged as a new generation of low-power deep neural networks, which is suitable to be implemented on low-power mobile/edge devices. As such devices have limited memory storage, neural pruning on SNNs has been widely explored in recent years. Most existing SNN pruning works focus on shallow SNNs (2~6 layers), however, deeper SNNs (>16 layers) are proposed by state-of-the-art SNN works, which is difficult to be compatible with the current SNN pruning work. To scale up a pruning technique towards deep SNNs, we investigate Lottery Ticket Hypothesis (LTH) which states that dense networks contain smaller subnetworks (i.e., winning tickets) that achieve comparable performance to the dense networks. Our studies on LTH reveal that the winning tickets consistently exist in deep SNNs across various datasets and architectures, providing up to 97% sparsity without huge performance degradation. However, the iterative searching process of LTH brings a huge training computational cost when combined with the multiple timesteps of SNNs. To alleviate such heavy searching cost, we propose Early-Time (ET) ticket where we find the important weight connectivity from a smaller number of timesteps. The proposed ET ticket can be seamlessly combined with a common pruning techniques for finding winning tickets, such as Iterative Magnitude Pruning (IMP) and Early-Bird (EB) tickets. Our experiment results show that the proposed ET ticket reduces search time by up to 38% compared to IMP or EB methods. Code is available at Github.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 64 of its 85 references that are in the map. First posted on arXiv 4 Jul 2022.

Builds on

64 of 85 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  9. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  10. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 626
  11. 2018Rethinking the Value of Network Pruningcited by 599
  12. 2016Learning Structured Sparsity in Deep Neural Networkscited by 541
38 more
  1. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  2. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 406
  3. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
  4. 2021Deep Residual Learning in Spiking Neural Networkscited by 331
  5. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 278
  6. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  7. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 266
  8. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 233
  9. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 207
  10. 2016Training Deep Spiking Neural Networks using Backpropagationcited by 205
  11. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 198
  12. 2018Long short-term memory and learning-to-learn in networks of spiking neuronscited by 181
  13. 2019Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networkscited by 160
  14. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 144
  15. 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 135
  16. 2019Enabling Spike-based Backpropagation for Training Deep Neural Network Architecturescited by 123
  17. 2021A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibrationcited by 117
  18. 2021Temporal-wise Attention Spiking Neural Networks for Event Streams Classificationcited by 106
  19. 2019One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizerscited by 102
  20. 2021Efficient Neuromorphic Signal Processing with Loihi 2cited by 96
  21. 2020Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratchcited by 80
  22. 2021Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networkscited by 78
  23. 2016Supervised learning based on temporal coding in spiking neural networkscited by 76
  24. 2019Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLPcited by 73
  25. 2021HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noisecited by 65
  26. 2022Neural Architecture Search for Spiking Neural Networkscited by 53
  27. 2020Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networkscited by 47
  28. 2017STDP Based Pruning of Connections and Weight Quantization in Spiking Neural Networks for Energy Efficient Recognitioncited by 46
  29. 2019Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagationcited by 45
  30. 2019Comprehensive SNN Compression Using ADMM Optimization and Activity Regularizationcited by 45
  31. 2021Efficient Lottery Ticket Finding: Less Data is Morecited by 44
  32. 2021Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsitycited by 44
  33. 2021Pruning of Deep Spiking Neural Networks through Gradient Rewiringcited by 43
  34. 2022Sparsity Winning Twice: Better Robust Generalization from More Efficient Trainingcited by 41
  35. 2020The Lottery Ticket Hypothesis for Object Recognitioncited by 34
  36. 2021On the Existence of Universal Lottery Ticketscited by 30
  37. 2020Brain-inspired global-local learning incorporated with neuromorphic computingcited by 28
  38. 2019Sparse Transfer Learning via Winning Lottery Ticketscited by 27

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 9; 2024: 8; 2025: 5; 2026: 1.

26 papers.

Sort:

  1. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  2. 2023Direct Learning-Based Deep Spiking Neural Networks: A Reviewcited by 31
  3. 2022Exploring Temporal Information Dynamics in Spiking Neural Networkscited by 24
  4. 2024A Survey of Lottery Ticket Hypothesiscited by 24
  5. 2023A Unified Framework for Soft Threshold Pruningcited by 12
  6. 2023Uncovering the Representation of Spiking Neural Networks Trained with Surrogate Gradientcited by 11
  7. 2024Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNNcited by 10
  8. 2024LoAS: Fully Temporal-Parallel Dataflow for Dual-Sparse Spiking Neural Networkscited by 8
  9. 2022Wearable-based Human Activity Recognition with Spatio-Temporal Spiking Neural Networkscited by 6
  10. 2022Hoyer regularizer is all you need for ultra low-latency spiking neural networkscited by 4
  11. 2023Probabilistic Modeling: Proving the Lottery Ticket Hypothesis in Spiking Neural Networkcited by 3
  12. 2024Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networkscited by 3
14 more
  1. 2025Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learningcited by 3
  2. 2023Highly Efficient SNNs for High-speed Object Detectioncited by 2
  3. 2023Activity Sparsity Complements Weight Sparsity for Efficient RNN Inferencecited by 2
  4. 2023Brain-Inspired Efficient Pruning: Exploiting Criticality in Spiking Neural Networkscited by 2
  5. 2024Direct Training Needs Regularisation: Anytime Optimal Inference Spiking Neural Networkcited by 2
  6. 2024Exploiting Heterogeneity in Timescales for Sparse Recurrent Spiking Neural Networks for Energy-Efficient Edge Computingcited by 2
  7. 2025Edge Intelligence with Spiking Neural Networkscited by 2
  8. 2023Gaining the Sparse Rewards by Exploring Lottery Tickets in Spiking Neural Networkcited by 1
  9. 2024Weight Sparsity Complements Activity Sparsity in Neuromorphic Language Modelscited by 1
  10. 2024When In-memory Computing Meets Spiking Neural Networks -- A Perspective on Device-Circuit-System-and-Algorithm Co-designcited by 1
  11. 2025Spiking Brain Compression: Post-Training Second-order Compression for Spiking Neural Networkscited by 0
  12. 2025SAFA-SNN: Sparsity-Aware On-Device Few-Shot Class-Incremental Learning with Fast-Adaptive Structure of Spiking Neural Networkcited by 0
  13. 2025MD-SNN: Membrane Potential-aware Distillation on Quantized Spiking Neural Networkcited by 0
  14. 2026Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliencycited by 0