# AerialFormer: Multi-resolution Transformer for Aerial Image Segmentation

arXiv: https://arxiv.org/abs/2306.06842 · HTML: https://mlsanity.com/paper/2306.06842/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 82 of its 121 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 30 of 82)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)

## Built on by (most cited first, 23 of 23)

- [MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining](https://mlsanity.com/paper/2403.13430/) (2024)
- [GiT: Towards Generalist Vision Transformer through Universal Language Interface](https://mlsanity.com/paper/2403.09394/) (2024)
- [Towards Open-Vocabulary Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2412.19492/) (2024)
- [OpenForest: A data catalogue for machine learning in forest monitoring](https://mlsanity.com/paper/2311.00277/) (2023)
- [SCORE: Scene Context Matters in Open-Vocabulary Remote Sensing Instance Segmentation](https://mlsanity.com/paper/2507.12857/) (2025)
- [Dynamic Dictionary Learning for Remote Sensing Image Segmentation](https://mlsanity.com/paper/2503.06683/) (2025)
- [Learning from Unlabelled Data with Transformers: Domain Adaptation for Semantic Segmentation of High Resolution Aerial Images](https://mlsanity.com/paper/2404.11299/) (2024)
- [CP2M: Clustered-Patch-Mixed Mosaic Augmentation for Aerial Image Segmentation](https://mlsanity.com/paper/2501.15389/) (2025)
- [Remote Sensing Semantic Segmentation Quality Assessment based on Vision Language Model](https://mlsanity.com/paper/2502.13990/) (2025)
- [Semantic Segmentation of Unmanned Aerial Vehicle Remote Sensing Images using SegFormer](https://mlsanity.com/paper/2410.01092/) (2024)
- [SolarFormer: Multi-scale Transformer for Solar PV Profiling](https://mlsanity.com/paper/2310.20057/) (2023)
- [Neural Embedding Compression For Efficient Multi-Task Earth Observation Modelling](https://mlsanity.com/paper/2403.17886/) (2024)
- [Heuristical Comparison of Vision Transformers Against Convolutional Neural Networks for Semantic Segmentation on Remote Sensing Imagery](https://mlsanity.com/paper/2411.09101/) (2024)
- [LightFormer: A lightweight and efficient decoder for remote sensing image segmentation](https://mlsanity.com/paper/2504.10834/) (2025)
- [ECMNet:Lightweight Semantic Segmentation with Efficient CNN-Mamba Network](https://mlsanity.com/paper/2506.08629/) (2025)
- [WakeupUrban: Unsupervised Semantic Segmentation of Mid-20th century Urban Landscapes with Satellite Imagery](https://mlsanity.com/paper/2506.09476/) (2025)
- [Stratify or Die: Rethinking Data Splits in Image Segmentation](https://mlsanity.com/paper/2509.21056/) (2025)
- [ChromaFormer: A Scalable and Accurate Transformer Architecture for Land Cover Classification](https://mlsanity.com/paper/2503.08534/) (2025)
- [Differential Morphological Profile Neural Networks for Semantic Segmentation](https://mlsanity.com/paper/2509.04268/) (2025)
- [BiCoR-Seg: Bidirectional Co-Refinement Framework for High-Resolution Remote Sensing Image Segmentation](https://mlsanity.com/paper/2512.20255/) (2025)
- [Mitigating Long-Tail Bias via Prompt-Controlled Diffusion Augmentation](https://mlsanity.com/paper/2602.04749/) (2026)
- [Hierarchical Adaptive Feature Refinement Network for VHR Remote Sensing Image Segmentation](https://mlsanity.com/paper/2608.15647/) (2026)
- [CRISP: Calibration-Aware Visual State Space Duality for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2608.23746/) (2026)

## Abstract (arXiv)

Aerial Image Segmentation is a top-down perspective semantic segmentation and has several challenging characteristics such as strong imbalance in the foreground-background distribution, complex background, intra-class heterogeneity, inter-class homogeneity, and tiny objects. To handle these problems, we inherit the advantages of Transformers and propose AerialFormer, which unifies Transformers at the contracting path with lightweight Multi-Dilated Convolutional Neural Networks (MD-CNNs) at the expanding path. Our AerialFormer is designed as a hierarchical structure, in which Transformer encoder outputs multi-scale features and MD-CNNs decoder aggregates information from the multi-scales. Thus, it takes both local and global contexts into consideration to render powerful representations and high-resolution segmentation. We have benchmarked AerialFormer on three common datasets including iSAID, LoveDA, and Potsdam. Comprehensive experiments and extensive ablation studies show that our proposed AerialFormer outperforms previous state-of-the-art methods with remarkable performance. Our source code will be publicly available upon acceptance.