mlsanityData to 7 Oct 2026

Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directions

arXiv 2409.02111 ↗First posted 19 Aug 2024cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Deep learning has revolutionized artificial intelligence (AI), achieving remarkable progress in fields such as computer vision, speech recognition, and natural language processing. Moreover, the recent success of large language models (LLMs) has fueled a surge in research on large-scale neural networks. However, the escalating demand for computing resources and energy consumption has prompted the search for energy-efficient alternatives. Inspired by the human brain, spiking neural networks (SNNs) promise energy-efficient computation with event-driven spikes. To provide future directions toward building energy-efficient large SNN models, we present a survey of existing methods for developing deep spiking neural networks, with a focus on emerging Spiking Transformers. Our main contributions are as follows: (1) an overview of learning methods for deep spiking neural networks, categorized by ANN-to-SNN conversion and direct training with surrogate gradients; (2) an overview of network architectures for deep spiking neural networks, categorized by deep convolutional neural networks (DCNNs) and Transformer architecture; and (3) a comprehensive comparison of state-of-the-art deep SNNs with a focus on emerging Spiking Transformers. We then further discuss and outline future directions toward large-scale SNNs.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 63 of its 117 references that are in the map. First posted on arXiv 19 Aug 2024.

Builds on

63 of 117 references.

References · 63

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  7. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  8. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
  9. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  10. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
  11. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
  12. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 251
38 more
  1. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 251
  2. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  3. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 222
  4. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 192
  5. 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 178
  6. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 174
  7. 2020RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networkcited by 141
  8. 2023Spike-driven Transformercited by 141
  9. 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 134
  10. 2022MetaFormer Baselines for Visioncited by 131
  11. 2023Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networkscited by 116
  12. 2021A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibrationcited by 111
  13. 2021Temporal-wise Attention Spiking Neural Networks for Event Streams Classificationcited by 106
  14. 2022Attention Spiking Neural Networkscited by 103
  15. 2024Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chipscited by 97
  16. 2023SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networkscited by 82
  17. 2020Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratchcited by 76
  18. 2021Advancing Spiking Neural Networks towards Deep Residual Learningcited by 76
  19. 2016Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computingcited by 68
  20. 2022GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 63
  21. 2022Neural Architecture Search for Spiking Neural Networkscited by 52
  22. 2023SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiationcited by 47
  23. 2020Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networkscited by 46
  24. 2024QKFormer: Hierarchical Spiking Transformer using Q-K Attentioncited by 46
  25. 2023Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANNcited by 42
  26. 2023Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillationcited by 41
  27. 2022AutoSNN: Towards Energy-Efficient Spiking Neural Networkscited by 40
  28. 2022Optimized Potential Initialization for Low-latency Spiking Neural Networkscited by 39
  29. 2022Masked Spiking Transformercited by 38
  30. 2024SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networkscited by 38
  31. 2023Membrane Potential Batch Normalization for Spiking Neural Networkscited by 35
  32. 2023Direct Learning-Based Deep Spiking Neural Networks: A Reviewcited by 31
  33. 2024SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spikingcited by 29
  34. 2024Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticketcited by 28
  35. 2023Darwin3: A large-scale neuromorphic chip with a Novel ISA and On-Chip Learningcited by 22
  36. 2022STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networkscited by 20
  37. 2023SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognitioncited by 20
  38. 2024Spiking-PhysFormer: Camera-Based Remote Photoplethysmography with Parallel Spike-driven Transformercited by 19

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2026SPARQ: Spiking Early-Exit Neural Networks for Energy-Efficient Edge AIcited by 1
  2. 2025Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networkscited by 0
  3. 2025DarwinWafer: A Wafer-Scale Neuromorphic Chipcited by 0
  4. 2026NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Frameworkcited by 0
  5. 2026LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learningcited by 0
  6. 2026SpikingVLA: Asynchronous Spiking Vision-Language-Action Modelscited by 0