# SSformer: A Lightweight Transformer for Semantic Segmentation

arXiv: https://arxiv.org/abs/2208.02034 · HTML: https://mlsanity.com/paper/2208.02034/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 17 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Understanding Convolution for Semantic Segmentation](https://mlsanity.com/paper/1702.08502/) (2017)

## Built on by (most cited first, 13 of 13)

- [PolypDB: A Curated Multi-Center Dataset for Development of AI Algorithms in Colonoscopy](https://mlsanity.com/paper/2409.00045/) (2024)
- [SAMIC: Segment Anything with In-Context Spatial Prompt Engineering](https://mlsanity.com/paper/2412.11998/) (2024)
- [B-Spine: Learning B-Spline Curve Representation for Robust and Interpretable Spinal Curvature Estimation](https://mlsanity.com/paper/2310.09603/) (2023)
- [Going Beyond U-Net: Assessing Vision Transformers for Semantic Segmentation in Microscopy Image Analysis](https://mlsanity.com/paper/2409.16940/) (2024)
- [ContextFormer: Redefining Efficiency in Semantic Segmentation](https://mlsanity.com/paper/2501.19255/) (2025)
- [An Efficient Semantic Segmentation Decoder for In-Car or Distributed Applications](https://mlsanity.com/paper/2510.16747/) (2025)
- [Weakly Supervised Camouflaged Object Detection Based on the SAM Model and Mask Guidance](https://mlsanity.com/paper/2605.25385/) (2026)
- [MUSTER: A Multi-scale Transformer-based Decoder for Semantic Segmentation](https://mlsanity.com/paper/2211.13928/) (2022)
- [DSNet: a simple yet efficient network with dual-stream attention for lesion segmentation](https://mlsanity.com/paper/2211.16950/) (2022)
- [Large-Scale Multi-Center CT and MRI Segmentation of Pancreas with Deep Learning](https://mlsanity.com/paper/2405.12367/) (2024)
- [DFR: A Decompose-Fuse-Reconstruct Framework for Multi-Modal Few-Shot Segmentation](https://mlsanity.com/paper/2507.16736/) (2025)
- [WaveSeg: Enhancing Segmentation Precision via High-Frequency Prior and Mamba-Driven Spectrum Decomposition](https://mlsanity.com/paper/2510.21079/) (2025)
- [APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms](https://mlsanity.com/paper/2606.30577/) (2026)

## Abstract (arXiv)

It is well believed that Transformer performs better in semantic segmentation compared to convolutional neural networks. Nevertheless, the original Vision Transformer may lack of inductive biases of local neighborhoods and possess a high time complexity. Recently, Swin Transformer sets a new record in various vision tasks by using hierarchical architecture and shifted windows while being more efficient. However, as Swin Transformer is specifically designed for image classification, it may achieve suboptimal performance on dense prediction-based segmentation task. Further, simply combing Swin Transformer with existing methods would lead to the boost of model size and parameters for the final segmentation model. In this paper, we rethink the Swin Transformer for semantic segmentation, and design a lightweight yet effective transformer model, called SSformer. In this model, considering the inherent hierarchical design of Swin Transformer, we propose a decoder to aggregate information from different layers, thus obtaining both local and global attentions. Experimental results show the proposed SSformer yields comparable mIoU performance with state-of-the-art models, while maintaining a smaller model size and lower compute.