mlsanityData to 13 Oct 2026

Exploring Temporal Information Dynamics in Spiking Neural Networks

arXiv 2211.14406 ↗First posted 26 Nov 2022cs.AI cs.NEcited by 24

Not on the map. Why?

Abstract (arXiv)

Most existing Spiking Neural Network (SNN) works state that SNNs may utilize temporal information dynamics of spikes. However, an explicit analysis of temporal information dynamics is still missing. In this paper, we ask several important questions for providing a fundamental understanding of SNNs: What are temporal information dynamics inside SNNs? How can we measure the temporal information dynamics? How do the temporal information dynamics affect the overall learning performance? To answer these questions, we estimate the Fisher Information of the weights to measure the distribution of temporal information during training in an empirical manner. Surprisingly, as training goes on, Fisher information starts to concentrate in the early timesteps. After training, we observe that information becomes highly concentrated in earlier few timesteps, a phenomenon we refer to as temporal information concentration. We observe that the temporal information concentration phenomenon is a common learning feature of SNNs by conducting extensive experiments on various configurations such as architecture, dataset, optimization strategy, time constant, and timesteps. Furthermore, to reveal how temporal information concentration affects the performance of SNNs, we design a loss function to change the trend of temporal information. We find that temporal information concentration is crucial to building a robust SNN but has little effect on classification accuracy. Finally, we propose an efficient iterative pruning method based on our observation on temporal information concentration. Code is available at https://github.com/Intelligent-Computing-Lab-Yale/Exploring-Temporal-Information-Dynamics-in-Spiking-Neural-Networks.

Key facts. Cited by 24 papers in the map (data through 13 Oct 2026). Builds on 35 of its 60 references that are in the map. First posted on arXiv 26 Nov 2022.

Builds on

35 of 60 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,713
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,379
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,753
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,328
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,017
  9. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,344
  10. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 787
  11. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 617
  12. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 403
23 more
  1. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
  2. 2021Deep Residual Learning in Spiking Neural Networkscited by 329
  3. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 275
  4. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 243
  5. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 231
  6. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 196
  7. 2019Enabling Spike-based Backpropagation for Training Deep Neural Network Architecturescited by 122
  8. 2021A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibrationcited by 115
  9. 2017Fisher-Rao Metric, Geometry, and Complexity of Neural Networkscited by 100
  10. 2020Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networkscited by 100
  11. 2018Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approachcited by 90
  12. 2020Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratchcited by 78
  13. 2016Supervised learning based on temporal coding in spiking neural networkscited by 76
  14. 2022Neural Architecture Search for Spiking Neural Networkscited by 53
  15. 2020Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activationscited by 48
  16. 2019Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagationcited by 45
  17. 2019Spiking Neural Networks and Online Learning: An Overview and Perspectivescited by 41
  18. 2020Exploring Adversarial Attack in Spiking Neural Networks with Spike-Compatible Gradientcited by 31
  19. 2021Visual Explanations from Spiking Neural Networks using Interspike Intervalscited by 26
  20. 2022Exploring Lottery Ticket Hypothesis in Spiking Neural Networkscited by 26
  21. 2021On the Variance of the Fisher Information for Deep Learningcited by 22
  22. 2021Spatio-Temporal Pruning and Quantization for Low-latency Spiking Neural Networkscited by 20
  23. 2021One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latencycited by 17

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 5; 2024: 6; 2025: 11; 2026: 2.

24 papers.

Sort:

By area:

  1. 2024Shrinking Your TimeStep: Towards Low-Latency Neuromorphic Object Recognition with Spiking Neural Networkcited by 19
  2. 2024EAS-SNN: End-to-End Adaptive Sampling and Representation for Event-based Detection with Recurrent Spiking Neural Networkscited by 16
  3. 2024TIM: An Efficient Temporal Interaction Module for Spiking Transformercited by 11
  4. 2023Best of Both Worlds: Hybrid SNN-ANN Architecture for Event-based Optical Flow Estimationcited by 4
  5. 2025Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexitycited by 4
  6. 2023Multi-scale Evolutionary Neural Architecture Search for Deep Spiking Neural Networkscited by 3
  7. 2024ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognitioncited by 3
  8. 2023Deep Unsupervised Learning Using Spike-Timing-Dependent Plasticitycited by 2
  9. 2023Brain-Inspired Efficient Pruning: Exploiting Criticality in Spiking Neural Networkscited by 2
  10. 2024Benchmarking Spiking Neural Network Learning Methods with Varying Localitycited by 2
  11. 2024SFedCA: Credit Assignment-Based Active Client Selection Strategy for Spiking Federated Learningcited by 2
  12. 2025Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocationcited by 2
12 more
  1. 2025SPARTA: Advancing Sparse Attention in Spiking Neural Networks via Spike-Timing-Based Prioritizationcited by 2
  2. 2025Synergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillerscited by 2
  3. 2025Spiking Neural Networks: The Future of Brain-Inspired Computingcited by 2
  4. 2023Brain-inspired Evolutionary Architectures for Spiking Neural Networkscited by 1
  5. 2025Dynamic Graph Condensationcited by 1
  6. 2025Spatio-Temporal Pruning for Compressed Spiking Large Language Modelscited by 1
  7. 2026PTS-SNN: A Prompt-Tuned Temporal Shift Spiking Neural Networks for Efficient Speech Emotion Recognitioncited by 1
  8. 2025Temporal Regularization Training: Unleashing the Potential of Spiking Neural Networkscited by 0
  9. 2025Adaptive Time-step Training for Enhancing Spike-Based Neural Radiance Fieldscited by 0
  10. 2025TDSNNs: Competitive Topographic Deep Spiking Neural Networks for Visual Cortex Modelingcited by 0
  11. 2025SDSNN: A Single-Timestep Spiking Neural Network with Self-Dropping Neuron and Bayesian Optimizationcited by 0
  12. 2026Controllable Stochastic Quantization Encoding for Adversarially Robust Spiking Neural Networkscited by 0