mlsanityData to 7 Oct 2026

Rethinking Spiking Neural Networks from an Ensemble Learning Perspective

arXiv 2502.14218 ↗First posted 20 Feb 2025cs.LG cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Spiking neural networks (SNNs) exhibit superior energy efficiency but suffer from limited performance. In this paper, we consider SNNs as ensembles of temporal subnetworks that share architectures and weights, and highlight a crucial issue that affects their performance: excessive differences in initial states (neuronal membrane potentials) across timesteps lead to unstable subnetwork outputs, resulting in degraded performance. To mitigate this, we promote the consistency of the initial membrane potential distribution and output through membrane potential smoothing and temporally adjacent subnetwork guidance, respectively, to improve overall stability and performance. Moreover, membrane potential smoothing facilitates forward propagation of information and backward propagation of gradients, mitigating the notorious temporal gradient vanishing problem. Our method requires only minimal modification of the spiking neurons without adapting the network structure, making our method generalizable and showing consistent performance gains in 1D speech, 2D object, and 3D point cloud recognition tasks. In particular, on the challenging CIFAR10-DVS dataset, we achieved 83.20% accuracy with only four timesteps. This provides valuable insights into unleashing the potential of SNNs.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 49 of its 75 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

49 of 75 references.

References · 49

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
  4. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  5. 2018Born Again Neural Networkscited by 539
  6. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  7. 2018Essentially No Barriers in Neural Network Energy Landscapecited by 328
  8. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
  9. 2015Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccadescited by 308
  10. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  11. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
  12. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 174
37 more
  1. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 163
  2. 2023Spike-driven Transformercited by 141
  3. 2019The Heidelberg spiking datasets for the systematic evaluation of spiking neural networkscited by 135
  4. 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 134
  5. 2023Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networkscited by 116
  6. 2024Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chipscited by 97
  7. 2023Deep Directly-Trained Spiking Neural Networks for Object Detectioncited by 76
  8. 2024Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detectioncited by 65
  9. 2019On Connected Sublevel Sets in Deep Learningcited by 51
  10. 2023Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependenciescited by 41
  11. 2024SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networkscited by 38
  12. 2023Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networkscited by 37
  13. 2023Ternary Spike: Learning Ternary Spikes for Spiking Neural Networkscited by 36
  14. 2023Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorizationcited by 34
  15. 2023Spiking PointNet: Spiking Neural Networks for Point Cloudscited by 34
  16. 2022TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networkscited by 33
  17. 2023ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classificationcited by 33
  18. 2020Hydra: Preserving Ensemble Diversity for Model Distillationcited by 31
  19. 2023RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networkscited by 27
  20. 2022Training Spiking Neural Networks with Local Tandem Learningcited by 22
  21. 2023TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modellingcited by 21
  22. 2024CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 21
  23. 2023DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networkscited by 20
  24. 2022Weighted Ensemble Self-Supervised Learningcited by 19
  25. 2024Shrinking Your TimeStep: Towards Low-Latency Neuromorphic Object Recognition with Spiking Neural Networkcited by 19
  26. 2024SpikeZIP-TF: Conversion is All You Need for Transformer-based SNNcited by 16
  27. 2023Temporal Knowledge Sharing enable Spiking Neural Network Learning from Past and Futurecited by 10
  28. 2024Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNNcited by 10
  29. 2023An Efficient Knowledge Transfer Strategy for Spiking Neural Networks from Static to Event Domaincited by 9
  30. 2024Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networkscited by 9
  31. 2024Autaptic Synaptic Circuit Enhances Spatio-temporal Predictive Learning of Spiking Neural Networkscited by 8
  32. 2022Ensembles of Compact, Region-specific & Regularized Spiking Neural Networks for Scalable Place Recognitioncited by 7
  33. 2023Addressing the speed-accuracy simulation trade-off for adaptive spiking neuronscited by 4
  34. 2020Ensembles of Spiking Neural Networkscited by 3
  35. 2023Temporal Contrastive Learning for Spiking Neural Networkscited by 3
  36. 2024Temporal Reversal Regularization for Spiking Neural Networks: Hybrid Spatio-Temporal Invariance for Generalizationcited by 2
  37. 2022Learning from Data with Noisy Labels Using Temporal Self-Ensemblecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 3.

12 papers.

  1. 2025Edge Intelligence with Spiking Neural Networkscited by 2
  2. 2025AR-LIF: Adaptive reset leaky integrate-and-fire neuron for spiking neural networkscited by 2
  3. 2025Synergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillerscited by 2
  4. 2025Exploring the Potentials of Spiking Neural Networks for Image Derainingcited by 2
  5. 2025TDFormer: A Top-Down Attention-Controlled Spiking Transformercited by 0
  6. 2025Boosting the Robustness-Accuracy Trade-off of SNNs by Robust Temporal Self-Ensemblecited by 0
  7. 2025HetSyn: Versatile Timescale Integration in Spiking Neural Networks via Heterogeneous Synapsescited by 0
  8. 2025SpikeMatch: Semi-Supervised Learning with Temporal Dynamics of Spiking Neural Networkscited by 0
  9. 2025Stabilizing Direct Training of Spiking Neural Networks: Membrane Potential Initialization and Threshold-robust Surrogate Gradientcited by 0
  10. 2026Stable Spike: Dual Consistency Optimization via Bitwise AND Operations for Spiking Neural Networkscited by 0
  11. 2026Ge2mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformercited by 0
  12. 2026Not All Timesteps Matter Equally: Selective Alignment Knowledge Distillation for Spiking Neural Networkscited by 0