# FSTA-SNN:Frequency-based Spatial-Temporal Attention Module for Spiking Neural Networks

arXiv: https://arxiv.org/abs/2501.14744 · HTML: https://mlsanity.com/paper/2501.14744/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 15 Dec 2024.

## Builds on (most cited first, 22 of 22)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Attention Mechanisms in Computer Vision: A Survey](https://mlsanity.com/paper/2111.07624/) (2021)
- [Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting](https://mlsanity.com/paper/2202.11946/) (2022)
- [FcaNet: Frequency Channel Attention Networks](https://mlsanity.com/paper/2012.11879/) (2020)
- [Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring](https://mlsanity.com/paper/2211.12250/) (2022)
- [RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network](https://mlsanity.com/paper/2003.01811/) (2020)
- [Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation](https://mlsanity.com/paper/2005.01807/) (2020)
- [Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks](https://mlsanity.com/paper/2103.12593/) (2021)
- [Temporal-wise Attention Spiking Neural Networks for Event Streams Classification](https://mlsanity.com/paper/2107.11711/) (2021)
- [Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation](https://mlsanity.com/paper/2205.00459/) (2022)
- [SpectFormer: Frequency and Attention is what you need in a Vision Transformer](https://mlsanity.com/paper/2304.06446/) (2023)
- [Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks](https://mlsanity.com/paper/2007.01204/) (2020)
- [Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization](https://mlsanity.com/paper/1911.00822/) (2019)
- [Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillation](https://mlsanity.com/paper/2304.05627/) (2023)
- [Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization](https://mlsanity.com/paper/2305.02099/) (2023)
- [Training Spiking Neural Networks with Local Tandem Learning](https://mlsanity.com/paper/2210.04532/) (2022)
- [Spatial-Frequency Attention for Image Denoising](https://mlsanity.com/paper/2302.13598/) (2023)

## Built on by (most cited first, 10 of 10)

- [TS-SNN: Temporal Shift Module for Spiking Neural Networks](https://mlsanity.com/paper/2505.04165/) (2025)
- [Edge Intelligence with Spiking Neural Networks](https://mlsanity.com/paper/2507.14069/) (2025)
- [Head-Tail-Aware KL Divergence in Knowledge Distillation for Spiking Neural Networks](https://mlsanity.com/paper/2504.20445/) (2025)
- [Fire on Motion: Optimizing Video Pass-bands for Efficient Spiking Action Recognition](https://mlsanity.com/paper/2601.22675/) (2026)
- [ChronoSpike: An Adaptive Spiking Graph Neural Network for Dynamic Graphs](https://mlsanity.com/paper/2602.01124/) (2026)
- [Vision SmolMamba: Spike-Guided Token Pruning for Energy-Efficient Spiking State-Space Vision Models](https://mlsanity.com/paper/2604.25570/) (2026)
- [Stable Spike: Dual Consistency Optimization via Bitwise AND Operations for Spiking Neural Networks](https://mlsanity.com/paper/2603.11676/) (2026)
- [Learnable Burst Quantization for Expressive and Efficient Spiking Neural Networks](https://mlsanity.com/paper/2604.25688/) (2026)
- [SAFformer:Improving Spiking Transformer via Active Predictive Filtering](https://mlsanity.com/paper/2605.08270/) (2026)
- [ORBITALIF: An Efficient Spiking Federated Learning Framework for Onboard Cloud Removal](https://mlsanity.com/paper/2608.24073/) (2026)

## Abstract (arXiv)

Spiking Neural Networks (SNNs) are emerging as a promising alternative to Artificial Neural Networks (ANNs) due to their inherent energy efficiency. Owing to the inherent sparsity in spike generation within SNNs, the in-depth analysis and optimization of intermediate output spikes are often neglected. This oversight significantly restricts the inherent energy efficiency of SNNs and diminishes their advantages in spatiotemporal feature extraction, resulting in a lack of accuracy and unnecessary energy expenditure. In this work, we analyze the inherent spiking characteristics of SNNs from both temporal and spatial perspectives. In terms of spatial analysis, we find that shallow layers tend to focus on learning vertical variations, while deeper layers gradually learn horizontal variations of features. Regarding temporal analysis, we observe that there is not a significant difference in feature learning across different time steps. This suggests that increasing the time steps has limited effect on feature learning. Based on the insights derived from these analyses, we propose a Frequency-based Spatial-Temporal Attention (FSTA) module to enhance feature learning in SNNs. This module aims to improve the feature learning capabilities by suppressing redundant spike features.The experimental results indicate that the introduction of the FSTA module significantly reduces the spike firing rate of SNNs, demonstrating superior performance compared to state-of-the-art baselines across multiple datasets.