# UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagery

arXiv: https://arxiv.org/abs/2109.08937 · HTML: https://mlsanity.com/paper/2109.08937/

## Key facts

- Cited by 167 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 104 references that are in the map.
- First posted on arXiv 18 Sep 2021.

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [Bottleneck Transformers for Visual Recognition](https://mlsanity.com/paper/2101.11605/) (2021)
- [Remote Sensing Image Change Detection with Transformers](https://mlsanity.com/paper/2103.00208/) (2021)
- [BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation](https://mlsanity.com/paper/2004.02147/) (2020)
- [ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data](https://mlsanity.com/paper/1904.00592/) (2019)
- [Co-Scale Conv-Attentional Image Transformers](https://mlsanity.com/paper/2104.06399/) (2021)
- [Asymmetric Non-local Neural Networks for Semantic Segmentation](https://mlsanity.com/paper/1908.07678/) (2019)

## Built on by (most cited first, 30 of 167)

- [Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model](https://mlsanity.com/paper/2208.03987/) (2022)
- [OpenEarthMap: A Benchmark Dataset for Global High-Resolution Land Cover Mapping](https://mlsanity.com/paper/2210.10732/) (2022)
- [MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining](https://mlsanity.com/paper/2403.13430/) (2024)
- [A Billion-scale Foundation Model for Remote Sensing Images](https://mlsanity.com/paper/2304.05215/) (2023)
- [Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks](https://mlsanity.com/paper/2309.16499/) (2023)
- [RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent](https://mlsanity.com/paper/2406.07089/) (2024)
- [Text2Seg: Remote Sensing Image Semantic Segmentation via Text-Guided Visual Foundation Models](https://mlsanity.com/paper/2304.10597/) (2023)
- [LSKNet: A Foundation Lightweight Backbone for Remote Sensing](https://mlsanity.com/paper/2403.11735/) (2024)
- [SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints](https://mlsanity.com/paper/2312.02464/) (2023)
- [RS3Mamba: Visual State Space Model for Remote Sensing Images Semantic Segmentation](https://mlsanity.com/paper/2404.02457/) (2024)
- [CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2410.22629/) (2024)
- [What a MESS: Multi-Domain Evaluation of Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2306.15521/) (2023)
- [Graph Information Bottleneck for Remote Sensing Segmentation](https://mlsanity.com/paper/2312.02545/) (2023)
- [AerialFormer: Multi-resolution Transformer for Aerial Image Segmentation](https://mlsanity.com/paper/2306.06842/) (2023)
- [Building extraction with vision transformer](https://mlsanity.com/paper/2111.15637/) (2021)
- [CM-UNet: Hybrid CNN-Mamba UNet for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2405.10530/) (2024)
- [RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation](https://mlsanity.com/paper/2504.03166/) (2025)
- [RingMo-Aerial: An Aerial Remote Sensing Foundation Model With Affine Transformation Contrastive Learning](https://mlsanity.com/paper/2409.13366/) (2024)
- [A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2410.11160/) (2024)
- [Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Experts](https://mlsanity.com/paper/2412.04220/) (2024)
- [HybridMIM: A Hybrid Masked Image Modeling Framework for 3D Medical Image Segmentation](https://mlsanity.com/paper/2303.10333/) (2023)
- [Decomposition-based Unsupervised Domain Adaptation for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2404.04531/) (2024)
- [PyramidMamba: Rethinking Pyramid Feature Fusion with Selective Space State Model for Semantic Segmentation of Remote Sensing Imagery](https://mlsanity.com/paper/2406.10828/) (2024)
- [Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensing](https://mlsanity.com/paper/2310.12452/) (2023)
- [Open-Vocabulary Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2409.07683/) (2024)
- [RS-vHeat: Heat Conduction Guided Efficient Remote Sensing Foundation Model](https://mlsanity.com/paper/2411.17984/) (2024)
- [Prototype-Driven Structure Synergy Network for Remote Sensing Images Segmentation](https://mlsanity.com/paper/2508.04022/) (2025)
- [The Potential of Visual ChatGPT For Remote Sensing](https://mlsanity.com/paper/2304.13009/) (2023)
- [LWGANet: Addressing Spatial and Channel Redundancy in Remote Sensing Visual Tasks with Light-Weight Grouped Attention](https://mlsanity.com/paper/2501.10040/) (2025)
- [Generalization Boosted Adapter for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2409.08468/) (2024)

## Abstract (arXiv)

Semantic segmentation of remotely sensed urban scene images is required in a wide range of practical applications, such as land cover mapping, urban change detection, environmental protection, and economic assessment.Driven by rapid developments in deep learning technologies, the convolutional neural network (CNN) has dominated semantic segmentation for many years. CNN adopts hierarchical feature representation, demonstrating strong capabilities for local information extraction. However, the local property of the convolution layer limits the network from capturing the global context. Recently, as a hot topic in the domain of computer vision, Transformer has demonstrated its great potential in global information modelling, boosting many vision-related tasks such as image classification, object detection, and particularly semantic segmentation. In this paper, we propose a Transformer-based decoder and construct a UNet-like Transformer (UNetFormer) for real-time urban scene segmentation. For efficient segmentation, the UNetFormer selects the lightweight ResNet18 as the encoder and develops an efficient global-local attention mechanism to model both global and local information in the decoder. Extensive experiments reveal that our method not only runs faster but also produces higher accuracy compared with state-of-the-art lightweight models. Specifically, the proposed UNetFormer achieved 67.8% and 52.4% mIoU on the UAVid and LoveDA datasets, respectively, while the inference speed can achieve up to 322.4 FPS with a 512x512 input on a single NVIDIA GTX 3090 GPU. In further exploration, the proposed Transformer-based decoder combined with a Swin Transformer encoder also achieves the state-of-the-art result (91.3% F1 and 84.1% mIoU) on the Vaihingen dataset. The source code will be freely available at https://github.com/WangLibo1995/GeoSeg.