mlsanityData to 7 Oct 2026

DTA: Dual Temporal-channel-wise Attention for Spiking Neural Networks

arXiv 2503.10052 ↗First posted 13 Mar 2025cs.CV cs.AI cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Spiking Neural Networks (SNNs) present a more energy-efficient alternative to Artificial Neural Networks (ANNs) by harnessing spatio-temporal dynamics and event-driven spikes. Effective utilization of temporal information is crucial for SNNs, leading to the exploration of attention mechanisms to enhance this capability. Conventional attention operations either apply identical operation or employ non-identical operations across target dimensions. We identify that these approaches provide distinct perspectives on temporal information. To leverage the strengths of both operations, we propose a novel Dual Temporal-channel-wise Attention (DTA) mechanism that integrates both identical/non-identical attention strategies. To the best of our knowledge, this is the first attempt to concentrate on both the correlation and dependency of temporal-channel using both identical and non-identical attention operations. Experimental results demonstrate that the DTA mechanism achieves state-of-the-art performance on both static datasets (CIFAR10, CIFAR100, ImageNet-1k) and dynamic dataset (CIFAR10-DVS), elevating spike representation and capturing complex temporal-channel relationship. We open-source our code: https://github.com/MnJnKIM/DTA-SNN.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 55 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

31 of 55 references.

References · 31

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  2. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  3. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  4. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 340
  5. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
  6. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  7. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
  8. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  9. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 222
  10. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 192
  11. 2020RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networkcited by 141
  12. 2023Spike-driven Transformercited by 141
19 more
  1. 2021A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibrationcited by 111
  2. 2021Temporal-wise Attention Spiking Neural Networks for Event Streams Classificationcited by 106
  3. 2022Attention Spiking Neural Networkscited by 103
  4. 2021Advancing Spiking Neural Networks towards Deep Residual Learningcited by 76
  5. 2021Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networkscited by 74
  6. 2020DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networkscited by 70
  7. 2022GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 63
  8. 2023Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANNcited by 42
  9. 2023Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependenciescited by 41
  10. 2023Reducing ANN-SNN Conversion Error through Residual Membrane Potentialcited by 39
  11. 2023Bridging the Gap between ANNs and SNNs by Calibrating Offset Spikescited by 38
  12. 2023Ternary Spike: Learning Ternary Spikes for Spiking Neural Networkscited by 36
  13. 2023Membrane Potential Batch Normalization for Spiking Neural Networkscited by 35
  14. 2022TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networkscited by 33
  15. 2023Gated Attention Coding for Training High-performance and Efficient Spiking Neural Networkscited by 26
  16. 2022Real Spike: Learning Real-valued Spikes for Spiking Neural Networkscited by 22
  17. 2024CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 21
  18. 2024SpikeZIP-TF: Conversion is All You Need for Transformer-based SNNcited by 16
  19. 2022Towards Lossless ANN-SNN Conversion under Ultra-Low Latency with Dual-Phase Optimizationcited by 13

Built on by

Not cited yet.