mlsanityData to 7 Oct 2026

Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks

arXiv 2103.12593 ↗First posted 12 Mar 2021cs.NE cs.LGcited by 113

Not on the map. Why?

Abstract (arXiv)

Inspired by more detailed modeling of biological neurons, Spiking neural networks (SNNs) have been investigated both as more biologically plausible and potentially more powerful models of neural computation, and also with the aim of extracting biological neurons' energy efficiency; the performance of such networks however has remained lacking compared to classical artificial neural networks (ANNs). Here, we demonstrate how a novel surrogate gradient combined with recurrent networks of tunable and adaptive spiking neurons yields state-of-the-art for SNNs on challenging benchmarks in the time-domain, like speech and gesture recognition. This also exceeds the performance of standard classical recurrent neural networks (RNNs) and approaches that of the best modern ANNs. As these SNNs exhibit sparse spiking, we show that they theoretically are one to three orders of magnitude more computationally efficient compared to RNNs with comparable performance. Together, this positions SNNs as an attractive solution for AI hardware implementations.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 22 of its 59 references that are in the map. First posted on arXiv 12 Mar 2021.

Builds on

22 of 59 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  4. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  5. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  6. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  7. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 340
  8. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  9. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 257
  10. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  11. 2018Long short-term memory and learning-to-learn in networks of spiking neuronscited by 179
  12. 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 178
10 more
  1. 2018Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNNcited by 142
  2. 2019The Heidelberg spiking datasets for the systematic evaluation of spiking neural networkscited by 135
  3. 2020Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networkscited by 59
  4. 2018A neural attention model for speech command recognitioncited by 44
  5. 2020Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networkscited by 43
  6. 2020Brain-Inspired Learning on Neuromorphic Substratescited by 35
  7. 2016STDP-based spiking deep convolutional neural networks for object recognitioncited by 24
  8. 2020Low-activity supervised convolutional spiking neural networks applied to speech commands recognitioncited by 19
  9. 2020TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devicescited by 16
  10. 2019Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDPcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 22; 2024: 42; 2025: 24; 2026: 13.

113 papers.

Sort:

By area:

  1. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
  2. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 163
  3. 2023Spike-driven Transformercited by 141
  4. 2022Attention Spiking Neural Networkscited by 103
  5. 2021Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networkscited by 99
  6. 2024Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chipscited by 97
  7. 2024Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Trainingcited by 57
  8. 2023Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipelinecited by 50
  9. 2023Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacingscited by 48
  10. 2021Accurate online training of dynamical spiking neural networks through Forward Propagation Through Timecited by 33
  11. 2023Direct Learning-Based Deep Spiking Neural Networks: A Reviewcited by 31
  12. 2024SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanismscited by 31
38 more
  1. 2024SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spikingcited by 29
  2. 2022STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networkscited by 20
  3. 2023Learning Long Sequences in Spiking Neural Networkscited by 19
  4. 2024SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Modelscited by 19
  5. 2024Spiker+: a framework for the generation of efficient Spiking Neural Networks FPGA accelerators for inference at the edgecited by 18
  6. 2022Loss shaping enhances exact gradient learning with Eventprop in spiking neural networkscited by 13
  7. 2023Inherent Redundancy in Spiking Neural Networkscited by 13
  8. 2023Co-learning synaptic delays, weights and adaptation in spiking neural networkscited by 12
  9. 2024Balanced Resonate-and-Fire Neuronscited by 12
  10. 2023Temporal Knowledge Sharing enable Spiking Neural Network Learning from Past and Futurecited by 10
  11. 2024Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learningcited by 10
  12. 2024FSTA-SNN:Frequency-based Spatial-Temporal Attention Module for Spiking Neural Networkscited by 10
  13. 2024DelGrad: Exact event-based gradients for training delays and weights on spiking neuromorphic hardwarecited by 9
  14. 2024SpGesture: Source-Free Domain-adaptive sEMG-based Gesture Recognition with Jaccard Attentive Spiking Neural Networkcited by 9
  15. 2022An Energy-Efficient Spiking Neural Network for Finger Velocity Decoding for Implantable Brain-Machine Interfacecited by 8
  16. 2024Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networkscited by 8
  17. 2024Autaptic Synaptic Circuit Enhances Spatio-temporal Predictive Learning of Spiking Neural Networkscited by 8
  18. 2024Brain-Inspired Online Adaptation for Remote Sensing with Spiking Neural Networkcited by 8
  19. 2023SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Featurescited by 6
  20. 2021Spike-inspired Rank Coding for Fast and Accurate Recurrent Neural Networkscited by 5
  21. 2023Efficient Uncertainty Estimation in Spiking Neural Networks via MC-dropoutcited by 5
  22. 2023Astrocyte-Enabled Advancements in Spiking Neural Networks for Large Language Modelingcited by 5
  23. 2024Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoderscited by 5
  24. 2024MK-SGN: A Spiking Graph Convolutional Network with Multimodal Fusion and Knowledge Distillation for Skeleton-based Action Recognitioncited by 5
  25. 2024Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoningcited by 5
  26. 2024Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Networkcited by 5
  27. 2024RSC-SNN: Exploring the Trade-off Between Adversarial Robustness and Accuracy in Spiking Neural Networks via Randomized Smoothing Codingcited by 5
  28. 2024DPSNN: Spiking Neural Network for Low-Latency Streaming Speech Enhancementcited by 5
  29. 2025The Resurrection of the ReLUcited by 5
  30. 2023A Unified Framework for Generative Data Augmentation: A Comprehensive Surveycited by 4
  31. 2023Fully Spiking Neural Network for Legged Robotscited by 4
  32. 2023Delayed Memory Unit: Modelling Temporal Dependency Through Delay Gatecited by 4
  33. 2023Addressing the speed-accuracy simulation trade-off for adaptive spiking neuronscited by 4
  34. 2023Brain-Inspired Spiking Neural Networks for Industrial Fault Diagnosis: A Survey, Challenges, and Opportunitiescited by 4
  35. 2024Global-Local Convolution with Spiking Neural Networks for Energy-efficient Keyword Spottingcited by 4
  36. 2025Scaling Up Resonate-and-Fire Networks for Fast Deep Learningcited by 4
  37. 2021A New Look at Spike-Timing-Dependent Plasticity Networks for Spatio-Temporal Feature Learningcited by 3
  38. 2022Stabilizing Spiking Neuron Trainingcited by 3

50 most cited of 113 All 113 (JSON)