# Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images

arXiv: https://arxiv.org/abs/2106.12413 · HTML: https://mlsanity.com/paper/2106.12413/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 71 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection](https://mlsanity.com/paper/1904.07392/) (2019)
- [ICNet for Real-Time Semantic Segmentation on High-Resolution Images](https://mlsanity.com/paper/1704.08545/) (2017)
- [ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data](https://mlsanity.com/paper/1904.00592/) (2019)
- [Fast-SCNN: Fast Semantic Segmentation Network](https://mlsanity.com/paper/1902.04502/) (2019)
- [PolyTransform: Deep Polygon Transformer for Instance Segmentation](https://mlsanity.com/paper/1912.02801/) (2019)
- [ResT: An Efficient Transformer for Visual Recognition](https://mlsanity.com/paper/2105.13677/) (2021)
- [Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks](https://mlsanity.com/paper/1711.08681/) (2017)
- [Multi-Attention-Network for Semantic Segmentation of Fine Resolution Remote Sensing Images](https://mlsanity.com/paper/2009.02130/) (2020)
- [A Novel Transformer Based Semantic Segmentation Scheme for Fine-Resolution Remote Sensing Images](https://mlsanity.com/paper/2104.12137/) (2021)
- [AlignSeg: Feature-Aligned Segmentation Networks](https://mlsanity.com/paper/2003.00872/) (2020)
- [Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection](https://mlsanity.com/paper/1612.01337/) (2016)
- [Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery](https://mlsanity.com/paper/1606.02585/) (2016)
- [Real-time Semantic Segmentation with Fast Attention](https://mlsanity.com/paper/2007.03815/) (2020)
- [Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models](https://mlsanity.com/paper/1803.06253/) (2018)
- [Multi-stage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Images](https://mlsanity.com/paper/2011.14302/) (2020)

## Built on by (most cited first, 28 of 28)

- [UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagery](https://mlsanity.com/paper/2109.08937/) (2021)
- [LSKNet: A Foundation Lightweight Backbone for Remote Sensing](https://mlsanity.com/paper/2403.11735/) (2024)
- [VDD: Varied Drone Dataset for Semantic Segmentation](https://mlsanity.com/paper/2305.13608/) (2023)
- [Graph Information Bottleneck for Remote Sensing Segmentation](https://mlsanity.com/paper/2312.02545/) (2023)
- [Building extraction with vision transformer](https://mlsanity.com/paper/2111.15637/) (2021)
- [Multi-Class Segmentation from Aerial Views using Recursive Noise Diffusion](https://mlsanity.com/paper/2212.00787/) (2022)
- [Hi-ResNet: Edge Detail Enhancement for High-Resolution Remote Sensing Segmentation](https://mlsanity.com/paper/2305.12691/) (2023)
- [Adaptive Frequency Enhancement Network for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2504.02647/) (2025)
- [Dynamic Dictionary Learning for Remote Sensing Image Segmentation](https://mlsanity.com/paper/2503.06683/) (2025)
- [MetaSegNet: Metadata-collaborative Vision-Language Representation Learning for Semantic Segmentation of Remote Sensing Images](https://mlsanity.com/paper/2312.12735/) (2023)
- [Remote Sensing Image Segmentation Using Vision Mamba and Multi-Scale Multi-Frequency Feature Fusion](https://mlsanity.com/paper/2410.05624/) (2024)
- [Vision Mamba in Remote Sensing: A Comprehensive Survey of Techniques, Applications and Outlook](https://mlsanity.com/paper/2505.00630/) (2025)
- [UNetMamba: An Efficient UNet-Like Mamba for Semantic Segmentation of High-Resolution Remote Sensing Images](https://mlsanity.com/paper/2408.11545/) (2024)
- [IRSAMap:Towards Large-Scale, High-Resolution Land Cover Map Vectorization](https://mlsanity.com/paper/2508.16272/) (2025)
- [Remote Sensing Semantic Segmentation Quality Assessment based on Vision Language Model](https://mlsanity.com/paper/2502.13990/) (2025)
- [M4Fog: A Global Multi-Regional, Multi-Modal, and Multi-Stage Dataset for Marine Fog Detection and Forecasting to Bridge Ocean and Atmosphere](https://mlsanity.com/paper/2406.13317/) (2024)
- [LightFormer: A lightweight and efficient decoder for remote sensing image segmentation](https://mlsanity.com/paper/2504.10834/) (2025)
- [Segment Any Vehicle: Semantic and Visual Context Driven SAM and A Benchmark](https://mlsanity.com/paper/2508.04260/) (2025)
- [Mapping and Classification of Trees Outside Forests using Deep Learning](https://mlsanity.com/paper/2510.25239/) (2025)
- [RemoteNet: Remote Sensing Image Segmentation Network based on Global-Local Information](https://mlsanity.com/paper/2302.13084/) (2023)
- [Global-Local Detail Guided Transformer for Sea Ice Recognition in Optical Remote Sensing Images](https://mlsanity.com/paper/2405.13197/) (2024)
- [UDHF2-Net: Uncertainty-diffusion-model-based High-Frequency TransFormer Network for Remotely Sensed Imagery Interpretation](https://mlsanity.com/paper/2406.16129/) (2024)
- [BAFNet: Bilateral Attention Fusion Network for Lightweight Semantic Segmentation of Urban Remote Sensing Images](https://mlsanity.com/paper/2409.10269/) (2024)
- [LKASeg:Remote-Sensing Image Semantic Segmentation with Large Kernel Attention and Full-Scale Skip Connections](https://mlsanity.com/paper/2410.10433/) (2024)
- [MFogHub: Bridging Multi-Regional and Multi-Satellite Data for Global Marine Fog Detection and Forecasting](https://mlsanity.com/paper/2505.10281/) (2025)
- [From Pixels to Images: A Structural Survey of Deep Learning Paradigms in Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2505.15147/) (2025)
- [AI in Agriculture: A Survey of Deep Learning Techniques for Crops, Fisheries and Livestock](https://mlsanity.com/paper/2507.22101/) (2025)
- [CRISP: Calibration-Aware Visual State Space Duality for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2608.23746/) (2026)

## Abstract (arXiv)

Semantic segmentation from very fine resolution (VFR) urban scene images plays a significant role in several application scenarios including autonomous driving, land cover classification, and urban planning, etc. However, the tremendous details contained in the VFR image, especially the considerable variations in scale and appearance of objects, severely limit the potential of the existing deep learning approaches. Addressing such issues represents a promising research field in the remote sensing community, which paves the way for scene-level landscape pattern analysis and decision making. In this paper, we propose a Bilateral Awareness Network which contains a dependency path and a texture path to fully capture the long-range relationships and fine-grained details in VFR images. Specifically, the dependency path is conducted based on the ResT, a novel Transformer backbone with memory-efficient multi-head self-attention, while the texture path is built on the stacked convolution operation. Besides, using the linear attention mechanism, a feature aggregation module is designed to effectively fuse the dependency features and texture features. Extensive experiments conducted on the three large-scale urban scene image segmentation datasets, i.e., ISPRS Vaihingen dataset, ISPRS Potsdam dataset, and UAVid dataset, demonstrate the effectiveness of our BANet. Specifically, a 64.6% mIoU is achieved on the UAVid dataset. Code is available at https://github.com/WangLibo1995/GeoSeg.