mlsanityData to 14 Oct 2026

Masked Spiking Transformer

arXiv 2210.01208 ↗First posted 3 Oct 2022cs.NEcited by 39

Not on the map. Why?

Abstract (arXiv)

The combination of Spiking Neural Networks (SNNs) and Transformers has attracted significant attention due to their potential for high energy efficiency and high-performance nature. However, existing works on this topic typically rely on direct training, which can lead to suboptimal performance. To address this issue, we propose to leverage the benefits of the ANN-to-SNN conversion method to combine SNNs and Transformers, resulting in significantly improved performance over existing state-of-the-art SNN models. Furthermore, inspired by the quantal synaptic failures observed in the nervous system, which reduces the number of spikes transmitted across synapses, we introduce a novel Masked Spiking Transformer (MST) framework that incorporates a Random Spike Masking (RSM) method to prune redundant spikes and reduce energy consumption without sacrificing performance. Our experimental results demonstrate that the proposed MST model achieves a significant reduction of 26.8% in power consumption when the masking ratio is 75% while maintaining the same level of performance as the unmasked model.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Oct 2022.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 13; 2025: 14; 2026: 7.

39 papers.

Sort:

By area:

  1. 2023Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Modelcited by 69
  2. 2024Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Trainingcited by 59
  3. 2024QKFormer: Hierarchical Spiking Transformer using Q-K Attentioncited by 49
  4. 2024Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraintcited by 24
  5. 2024A Survey: Collaborative Hardware and Software Design in the Era of Large Language Modelscited by 23
  6. 2023Chasing Day and Night: Towards Robust and Efficient All-Day Object Detection Guided by an Event Cameracited by 21
  7. 2024SpikeZIP-TF: Conversion is All You Need for Transformer-based SNNcited by 18
  8. 2024Learning A Spiking Neural Network for Efficient Image Derainingcited by 16
  9. 2024Autonomous Driving with Spiking Neural Networkscited by 13
  10. 2025Quantized Spike-driven Transformercited by 13
  11. 2024LM-HT SNN: Enhancing the Performance of SNN to ANN Counterpart through Learnable Multi-hierarchical Threshold Modelcited by 11
  12. 2025Differential Coding for Training-Free ANN-to-SNN Conversioncited by 10
27 more
  1. 2023Optimising Event-Driven Spiking Neural Network with Regularisation and Cutoffcited by 8
  2. 2024Spiking Neural Network as Adaptive Event Stream Slicercited by 8
  3. 2024Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directionscited by 7
  4. 2024Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networkcited by 6
  5. 2025Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networkscited by 6
  6. 2023RN-Net: Reservoir Nodes-Enabled Neuromorphic Vision Sensing Networkcited by 3
  7. 2025Spiking Neural Networks Need High Frequency Informationcited by 3
  8. 2024Brain-inspired Action Generation with Spiking Transformer Diffusion Policy Modelcited by 2
  9. 2025MSVIT: Improving Spiking Vision Transformer Using Multi-scale Attention Fusioncited by 2
  10. 2025Edge Intelligence with Spiking Neural Networkscited by 2
  11. 2026Neural Dynamics Self-Attention for Spiking Transformerscited by 2
  12. 2026Spiking Pyramid Wavelet Transformation for High-efficient and Low-energy Image Restorationcited by 2
  13. 2024SDformerFlow: Spatiotemporal swin spikeformer for event-based optical flow estimationcited by 1
  14. 2025Region Masking to Accelerate Video Processing on Neuromorphic Hardwarecited by 1
  15. 2025SpikeDerain: Unveiling Clear Videos from Rainy Sequences Using Color Spike Streamscited by 1
  16. 2025IML-Spikeformer: Input-aware Multi-Level Spiking Transformer for Speech Processingcited by 1
  17. 2026Error Amplification Limits ANN-to-SNN Conversion in Continuous Controlcited by 1
  18. 2023Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networkscited by 0
  19. 2025Improving vision-language alignment with graph spiking hybrid Networkscited by 0
  20. 2025Temporal Misalignment in ANN-SNN Conversion and Its Mitigation via Probabilistic Spiking Neuronscited by 0
  21. 2025Quantization Meets Spikes: Nearly Lossless Conversion at the First Timestep via Polarity Multi-Spike Mappingcited by 0
  22. 2025NSPDI-SNN: An efficient lightweight SNN based on nonlinear synaptic pruning and dendritic integrationcited by 0
  23. 2025CSDformer: A Conversion Method for Fully Spike-Driven Transformercited by 0
  24. 2026Ge2mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformercited by 0
  25. 2026SAFformer:Improving Spiking Transformer via Active Predictive Filteringcited by 0
  26. 2026A Neuromorphic Reinforcement Learning Framework for Efficient Pathfinding in Robotic Mobile Fulfillment Systemscited by 0
  27. 2026SCINTILLA-SNN: A Spiking Multi-Scale Selective Aggregation Network for Perineural Invasion Predictioncited by 0