# Spike2Former: Efficient Spiking Transformer for High-performance Image Segmentation

arXiv: https://arxiv.org/abs/2412.14587 · HTML: https://mlsanity.com/paper/2412.14587/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 19 Dec 2024.

## Builds on (most cited first, 23 of 23)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Spikformer: When Spiking Neural Network Meets Transformer](https://mlsanity.com/paper/2209.15425/) (2022)
- [Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips](https://mlsanity.com/paper/2404.03663/) (2024)
- [Advancing Spiking Neural Networks towards Deep Residual Learning](https://mlsanity.com/paper/2112.08954/) (2021)
- [Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detection](https://mlsanity.com/paper/2407.20708/) (2024)
- [Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training](https://mlsanity.com/paper/2411.16061/) (2024)
- [Beyond Classification: Directly Training Spiking Neural Networks for Semantic Segmentation](https://mlsanity.com/paper/2110.07742/) (2021)
- [Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline](https://mlsanity.com/paper/2304.06793/) (2023)
- [Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks](https://mlsanity.com/paper/2007.01204/) (2020)
- [PEM: Prototype-based Efficient MaskFormer for Image Segmentation](https://mlsanity.com/paper/2402.19422/) (2024)
- [A Spiking Neural Network for Image Segmentation](https://mlsanity.com/paper/2106.08921/) (2021)
- [You Only Learn One Query: Learning Unified Human Query for Single-Stage Multi-Person Multi-Task Human-Centric Perception](https://mlsanity.com/paper/2312.05525/) (2023)
- [Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control](https://mlsanity.com/paper/2303.11860/) (2023)
- [Mixed-Query Transformer: A Unified Image Segmentation Architecture](https://mlsanity.com/paper/2404.04469/) (2024)

## Built on by (most cited first, 19 of 19)

- [SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks](https://mlsanity.com/paper/2503.08703/) (2025)
- [SpikeTrack: A Spike-driven Framework for Efficient Visual Tracking](https://mlsanity.com/paper/2602.23963/) (2026)
- [Hardware, Algorithms, and Applications of the Neuromorphic Vision Sensor: a Review](https://mlsanity.com/paper/2504.08588/) (2025)
- [Edge Intelligence with Spiking Neural Networks](https://mlsanity.com/paper/2507.14069/) (2025)
- [Exploring the Potentials of Spiking Neural Networks for Image Deraining](https://mlsanity.com/paper/2512.02258/) (2025)
- [Efficient Spike-driven Transformer for High-performance Drone-View Geo-Localization](https://mlsanity.com/paper/2512.19365/) (2025)
- [Segment Any Vehicle: Semantic and Visual Context Driven SAM and A Benchmark](https://mlsanity.com/paper/2508.04260/) (2025)
- [Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrieval](https://mlsanity.com/paper/2603.26787/) (2026)
- [A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks](https://mlsanity.com/paper/2606.11236/) (2026)
- [SpikeVLA: Vision-Language-Action Models with Spiking Neural Networks](https://mlsanity.com/paper/2606.27807/) (2026)
- [SPEAR: Structured Pruning for Spiking Neural Networks via Synaptic Operation Estimation and Reinforcement Learning](https://mlsanity.com/paper/2507.02945/) (2025)
- [Toward Real-Time Surgical Scene Segmentation via a Spike-Driven Video Transformer with Spike-Informed Pretraining](https://mlsanity.com/paper/2512.21284/) (2025)
- [UIESNN: A Scale-Aware Spiking Network for Underwater Image Enhancement](https://mlsanity.com/paper/2605.08376/) (2026)
- [SpikF-GO: Spiking Fourier Graph Operators for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2606.13901/) (2026)
- [Robust PnP on a Neuromorphic Processor for Object Pose Estimation](https://mlsanity.com/paper/2607.16834/) (2026)
- [Rethinking Attention Locality in Spiking Transformers](https://mlsanity.com/paper/2608.08541/) (2026)
- [Spiking Local Interaction and Adaptive Complementary Fusion for Spiking Transformer](https://mlsanity.com/paper/2608.19238/) (2026)
- [SBMVTrack: Spike-Budgeted Multi-View Learning for Power-Efficient UAV Tracking](https://mlsanity.com/paper/2609.25503/) (2026)
- [SpikeOcc: Towards Resilient Multimodal 3D Occupancy Perception via Spiking Experts](https://mlsanity.com/paper/2610.15745/) (2026)

## Abstract (arXiv)

Spiking Neural Networks (SNNs) have a low-power advantage but perform poorly in image segmentation tasks. The reason is that directly converting neural networks with complex architectural designs for segmentation tasks into spiking versions leads to performance degradation and non-convergence. To address this challenge, we first identify the modules in the architecture design that lead to the severe reduction in spike firing, make targeted improvements, and propose Spike2Former architecture. Second, we propose normalized integer spiking neurons to solve the training stability problem of SNNs with complex architectures. We set a new state-of-the-art for SNNs in various semantic segmentation datasets, with a significant improvement of +12.7% mIoU and 5.0 efficiency on ADE20K, +14.3% mIoU and 5.2 efficiency on VOC2012, and +9.1% mIoU and 6.6 efficiency on CityScapes.