mlsanityData to 14 Oct 2026

Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies

arXiv 2304.12760 ↗First posted 25 Apr 2023cs.NE cs.CVcited by 41

Not on the map. Why?

Abstract (arXiv)

Vanilla spiking neurons in Spiking Neural Networks (SNNs) use charge-fire-reset neuronal dynamics, which can only be simulated serially and can hardly learn long-time dependencies. We find that when removing reset, the neuronal dynamics can be reformulated in a non-iterative form and parallelized. By rewriting neuronal dynamics without reset to a general formulation, we propose the Parallel Spiking Neuron (PSN), which generates hidden states that are independent of their predecessors, resulting in parallelizable neuronal dynamics and extremely high simulation speed. The weights of inputs in the PSN are fully connected, which maximizes the utilization of temporal information. To avoid the use of future inputs for step-by-step inference, the weights of the PSN can be masked, resulting in the masked PSN. By sharing weights across time-steps based on the masked PSN, the sliding PSN is proposed to handle sequences of varying lengths. We evaluate the PSN family on simulation speed and temporal/static data classification, and the results show the overwhelming advantage of the PSN family in efficiency and accuracy. To the best of our knowledge, this is the first study about parallelizing spiking neurons and can be a cornerstone for the spiking deep learning research. Our codes are available at urlhttps://github.com/fangwei123456/Parallel-Spiking-Neuron.

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). Builds on 43 of its 68 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

43 of 68 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  7. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  8. 2022A ConvNet for the 2020scited by 3,579
  9. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  10. 2017Random Erasing Data Augmentationcited by 1,239
  11. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 626
  12. 2017Convolutional Sequence to Sequence Learningcited by 552
31 more
  1. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 406
  2. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
  3. 2021Deep Residual Learning in Spiking Neural Networkscited by 331
  4. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 278
  5. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 271
  6. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 266
  7. 2018Deep Learning in Spiking Neural Networkscited by 263
  8. 2021TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentationcited by 227
  9. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 198
  10. 2020RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networkcited by 145
  11. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 144
  12. 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 139
  13. 2017Parallelizing Linear Recurrent Neural Nets Over Sequence Lengthcited by 127
  14. 2023Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networkscited by 122
  15. 2022Attention Spiking Neural Networkscited by 107
  16. 2022Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representationcited by 89
  17. 2020Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratchcited by 80
  18. 2021Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networkscited by 78
  19. 2016Neural Machine Translation in Linear Timecited by 66
  20. 2022GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 63
  21. 2019S4NN: temporal backpropagation for spiking neural networks with one spike per neuroncited by 49
  22. 2022Online Training Through Time for Spiking Neural Networkscited by 49
  23. 2020T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Codingcited by 46
  24. 2021Sparse Spiking Gradient Descentcited by 41
  25. 2023Reducing ANN-SNN Conversion Error through Residual Membrane Potentialcited by 40
  26. 2021Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learningcited by 27
  27. 2020BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learningcited by 24
  28. 2020Training Deep Spiking Neural Networkscited by 21
  29. 2022Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networkscited by 21
  30. 2023KLIF: An optimized spiking neuron unit for tuning surrogate gradient slope and membrane potentialcited by 6
  31. 2022A Time-to-first-spike Coding and Conversion Aware Training for Energy-Efficient Deep Spiking Neural Network Processor Designcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 16; 2025: 13; 2026: 11.

41 papers.

Sort:

By area:

  1. 2024Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Trainingcited by 59
  2. 2024Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Modelscited by 31
  3. 2024SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanismscited by 31
  4. 2024SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Modelscited by 19
  5. 2025Rethinking Spiking Neural Networks from an Ensemble Learning Perspectivecited by 12
  6. 2024LM-HT SNN: Enhancing the Performance of SNN to ANN Counterpart through Learnable Multi-hierarchical Threshold Modelcited by 11
  7. 2024SPikE-SSM: A Sparse, Precise, and Efficient Spiking State Space Model for Long Sequences Learningcited by 9
  8. 2024EZSR: Event-based Zero-Shot Recognitioncited by 8
  9. 2024Brain-Inspired Online Adaptation for Remote Sensing with Spiking Neural Networkcited by 8
  10. 2025Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexitycited by 5
  11. 2026Time Is All It Takes: Spike-Retiming Attacks on Event-Driven Spiking Neural Networkscited by 5
  12. 2025Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Timecited by 4
29 more
  1. 2024SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Networkcited by 3
  2. 2024Faster and Stronger: When ANN-SNN Conversion Meets Parallel Spiking Calculationcited by 2
  3. 2025Edge Intelligence with Spiking Neural Networkscited by 2
  4. 2025Dendritic Resonate-and-Fire Neuron for Effective and Efficient Long Sequence Modelingcited by 2
  5. 2026Modern Neuromorphic AI: From Intra-Token to Inter-Token Processingcited by 2
  6. 2026Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networkscited by 2
  7. 2024Weight Sparsity Complements Activity Sparsity in Neuromorphic Language Modelscited by 1
  8. 2024SDformerFlow: Spatiotemporal swin spikeformer for event-based optical flow estimationcited by 1
  9. 2024Time-independent Spiking Neuron via Membrane Potential Estimation for Efficient Spiking Neural Networkscited by 1
  10. 2025Efficient Training of Spiking Neural Networks by Spike-aware Data Pruningcited by 1
  11. 2026Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradientscited by 1
  12. 2023NiSNN-A: Non-iterative Spiking Neural Networks with Attention with Application to Motor Imagery EEG Classificationcited by 0
  13. 2024Topology Optimization of Random Memristors for Input-Aware Dynamic SNNcited by 0
  14. 2024Obtaining Optimal Spiking Neural Network in Sequence Learning via CRNN-SNN Conversioncited by 0
  15. 2024Network Representation Learning for Biophysical Neural Network Analysiscited by 0
  16. 2025DTA: Dual Temporal-channel-wise Attention for Spiking Neural Networkscited by 0
  17. 2025Revisiting Reset Mechanisms in Spiking Neural Networks for Sequential Modeling: Specialized Discretization for Binary Activated RNNcited by 0
  18. 2025A Scalable Hybrid Training Approach for Recurrent Spiking Neural Networkscited by 0
  19. 2025Optimization of Low-Latency Spiking Neural Networks Utilizing Historical Dynamics of Refractory Periodscited by 0
  20. 2025Adaptive Time-step Training for Enhancing Spike-Based Neural Radiance Fieldscited by 0
  21. 2025Stabilizing Direct Training of Spiking Neural Networks: Membrane Potential Initialization and Threshold-robust Surrogate Gradientcited by 0
  22. 2025Towards Reliable Evaluation of Adversarial Robustness for Spiking Neural Networkscited by 0
  23. 2026Robust Spiking Neural Networks Against Adversarial Attackscited by 0
  24. 2026Stable Spike: Dual Consistency Optimization via Bitwise AND Operations for Spiking Neural Networkscited by 0
  25. 2026Neuronal Self-Adaptation Enhances Capacity and Robustness of Representation in Spiking Neural Networkscited by 0
  26. 2026Ge2mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformercited by 0
  27. 2026ShiftLIF: Efficient Multi-Level Spiking Neurons with Power-of-Two Quantizationcited by 0
  28. 2026Noisy group neurons with synchronous resetting for high-performance spiking neural networkscited by 0
  29. 2026SpikeMoE: Brain-Inspired Competitive Routing for Flexible Spiking Mixture-of-Expertscited by 0