mlsanityData to 14 Oct 2026

High-performance deep spiking neural networks with 0.3 spikes per neuron

arXiv 2306.08744 ↗First posted 14 Jun 2023cs.NE cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Communication by rare, binary spikes is a key factor for the energy efficiency of biological brains. However, it is harder to train biologically-inspired spiking neural networks (SNNs) than artificial neural networks (ANNs). This is puzzling given that theoretical results provide exact mapping algorithms from ANNs to SNNs with time-to-first-spike (TTFS) coding. In this paper we analyze in theory and simulation the learning dynamics of TTFS-networks and identify a specific instance of the vanishing-or-exploding gradient problem. While two choices of SNN mappings solve this problem at initialization, only the one with a constant slope of the neuron membrane potential at threshold guarantees the equivalence of the training trajectory between SNNs and ANNs with rectified linear units. We demonstrate that training deep SNN models achieves the exact same performance as that of ANNs, surpassing previous SNNs on image classification datasets such as MNIST/Fashion-MNIST, CIFAR10/CIFAR100 and PLACES365. Our SNN accomplishes high-performance classification with less than 0.3 spikes per neuron, lending itself for an energy-efficient implementation. We show that fine-tuning SNNs with our robust gradient descent algorithm enables their optimization for hardware implementations with low latency and resilience to noise and quantization.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 33 of its 83 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

33 of 83 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  4. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  5. 2019Event-based Vision: A Surveycited by 1,016
  6. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 626
  7. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  8. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  9. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 184
  10. 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 182
  11. 2018Long short-term memory and learning-to-learn in networks of spiking neuronscited by 181
  12. 2023Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networkscited by 122
21 more
  1. 2019Convergence of Edge Computing and Deep Learning: A Comprehensive Surveycited by 120
  2. 2021A Study of Face Obfuscation in ImageNetcited by 83
  3. 2016Supervised learning based on temporal coding in spiking neural networkscited by 76
  4. 2021Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscited by 65
  5. 2020Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networkscited by 59
  6. 2019Fast and energy-efficient neuromorphic deep learning with first-spike timescited by 52
  7. 2017Gradient Descent for Spiking Neural Networkscited by 48
  8. 2019Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performancecited by 46
  9. 2019Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagationcited by 45
  10. 2016Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardwarecited by 40
  11. 2016STDP-based spiking deep convolutional neural networks for object recognitioncited by 24
  12. 2021Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottleneckscited by 23
  13. 2017Neuromorphic Hardware In The Loop: Training a Deep Spiking Network on the BrainScaleS Wafer-Scale Systemcited by 19
  14. 2018Demonstrating Advantages of Neuromorphic Computation: A Pilot Studycited by 14
  15. 2014Random Walk Initialization for Training Very Deep Feedforward Networkscited by 13
  16. 2022An Exact Mapping From ReLU Networks to Spiking Neural Networkscited by 12
  17. 2019Improving Noise Tolerance of Mixed-Signal Neural Networkscited by 9
  18. 2018On the choice of metric in gradient-based theories of brain functioncited by 8
  19. 2017Energy-Efficient Time-Domain Vector-by-Matrix Multiplier for Neurocomputing and Beyondcited by 5
  20. 2015Encoding Spike Patterns in Multilayer Spiking Neural Networkscited by 3
  21. 2021Training Energy-Efficient Deep Spiking Neural Networks with Time-to-First-Spike Codingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 8; 2026: 10.

22 papers.

Sort:

By area:

  1. 2024DelGrad: Exact event-based gradients for training delays and weights on spiking neuromorphic hardwarecited by 9
  2. 2024Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoderscited by 5
  3. 2025Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Timecited by 4
  4. 2024Deep Oscillatory Neural Networkcited by 3
  5. 2025Causal pieces: analysing and improving spiking neural networks piece by piececited by 2
  6. 2024Spiking Music: Audio Compression with Event Based Auto-encoderscited by 1
  7. 2025Sparse by Rule: Probability-Based N:M Pruning for Spiking Neural Networkscited by 1
  8. 2026Orthogonal Weight Modification Enhances Learning Scalability and Convergence Efficiency without Gradient Backpropagationcited by 1
  9. 2026Equivalence of approximation by networks of single- and multi-spike neuronscited by 1
  10. 2026Vision SmolMamba: Spike-Guided Token Pruning for Energy-Efficient Spiking State-Space Vision Modelscited by 1
  11. 2025Revisiting Reset Mechanisms in Spiking Neural Networks for Sequential Modeling: Specialized Discretization for Binary Activated RNNcited by 0
  12. 2025Frozen Backpropagation: Relaxing Weight Symmetry in Deep Spiking Neural Networkscited by 0
10 more
  1. 2025Adaptive Time-step Training for Enhancing Spike-Based Neural Radiance Fieldscited by 0
  2. 2025Complementary Learning System Empowers Online Continual Learning of Vehicle Motion Forecasting in Smart Citiescited by 0
  3. 2025Predictive Spike Timing Enables Distributed Shortest Path Computation in Spiking Neural Networkscited by 0
  4. 2026NEXUS: Bit-Exact ANN-to-SNN Equivalence via Neuromorphic Gate Circuits with Surrogate-Free Trainingcited by 0
  5. 2026Spark: Modular Spiking Neural Networkscited by 0
  6. 2026Wi-Spike: A Low-power WiFi Human Multi-action Recognition Model with Spiking Neural Networkscited by 0
  7. 2026Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networkscited by 0
  8. 2026Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretizationcited by 0
  9. 2026SAGE: Surrogate-gradient Adaptation via Attention-Guided Entropy for Spiking Transformerscited by 0
  10. 2026Large Language Models with At Most One Spike per Neuroncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022S3NN: Time Step Reduction of Spiking Surrogate Gradients for Training Energy Efficient Single-Step Spiking Neural Networks
  2. 2022tinySNN: Towards Memory- and Energy-Efficient Spiking Neural Networks
  3. 2023Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN
  4. 2023Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization
  5. 2023When Bio-Inspired Computing meets Deep Learning: Low-Latency, Accurate, & Energy-Efficient Spiking Neural Networks from Artificial Neural Networks
  6. 2025All in one timestep: Enhancing Sparsity and Energy efficiency in Multi-level Spiking Neural Networks
  7. 2024Deep activity propagation via weight initialization in spiking neural networks
  8. 2025Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocation