# Real-time Semantic Segmentation with Fast Attention

arXiv: https://arxiv.org/abs/2007.03815 · HTML: https://mlsanity.com/paper/2007.03815/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2020.

## Built on by (most cited first, 30 of 36)

- [UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagery](https://mlsanity.com/paper/2109.08937/) (2021)
- [Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes](https://mlsanity.com/paper/2101.06085/) (2021)
- [U-Net Transformer: Self and Cross Attention for Medical Image Segmentation](https://mlsanity.com/paper/2103.06104/) (2021)
- [Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation](https://mlsanity.com/paper/2203.01452/) (2022)
- [Both Style and Distortion Matter: Dual-Path Unsupervised Domain Adaptation for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2303.14360/) (2023)
- [Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2207.11860/) (2022)
- [LSKNet: A Foundation Lightweight Backbone for Remote Sensing](https://mlsanity.com/paper/2403.11735/) (2024)
- [DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange](https://mlsanity.com/paper/2108.06383/) (2021)
- [Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation](https://mlsanity.com/paper/2110.11062/) (2021)
- [AuxAdapt: Stable and Efficient Test-Time Adaptation for Temporally Consistent Video Semantic Segmentation](https://mlsanity.com/paper/2110.12369/) (2021)
- [Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images](https://mlsanity.com/paper/2106.12413/) (2021)
- [Local Memory Attention for Fast Video Semantic Segmentation](https://mlsanity.com/paper/2101.01715/) (2021)
- [ABCNet: Attentive Bilateral Contextual Network for Efficient Semantic Segmentation of Fine-Resolution Remote Sensing Images](https://mlsanity.com/paper/2102.02531/) (2021)
- [AttaNet: Attention-Augmented Network for Fast and Accurate Scene Parsing](https://mlsanity.com/paper/2103.05930/) (2021)
- [Computation-efficient Deep Learning for Computer Vision: A Survey](https://mlsanity.com/paper/2308.13998/) (2023)
- [Delta Distillation for Efficient Video Processing](https://mlsanity.com/paper/2203.09594/) (2022)
- [Hi-ResNet: Edge Detail Enhancement for High-Resolution Remote Sensing Segmentation](https://mlsanity.com/paper/2305.12691/) (2023)
- [On Efficient Real-Time Semantic Segmentation: A Survey](https://mlsanity.com/paper/2206.08605/) (2022)
- [Automatic universal taxonomies for multi-domain semantic segmentation](https://mlsanity.com/paper/2207.08445/) (2022)
- [Toward Edge-Efficient Dense Predictions with Synergistic Multi-Task Neural Architecture Search](https://mlsanity.com/paper/2210.01384/) (2022)
- [Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired](https://mlsanity.com/paper/2103.04136/) (2021)
- [Memory-Constrained Semantic Segmentation for Ultra-High Resolution UAV Imagery](https://mlsanity.com/paper/2310.04721/) (2023)
- [CSRNet: Cascaded Selective Resolution Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/2106.04400/) (2021)
- [A Deep Learning Driven Algorithmic Pipeline for Autonomous Navigation in Row-Based Crops](https://mlsanity.com/paper/2112.03816/) (2021)
- [S2-FPN: Scale-ware Strip Attention Guided Feature Pyramid Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/2206.07298/) (2022)
- [Simple and Efficient Architectures for Semantic Segmentation](https://mlsanity.com/paper/2206.08236/) (2022)
- [A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design](https://mlsanity.com/paper/2303.04315/) (2023)
- [Compressing Vision Transformers for Low-Resource Visual Learning](https://mlsanity.com/paper/2309.02617/) (2023)
- [2023 Low-Power Computer Vision Challenge (LPCVC) Summary](https://mlsanity.com/paper/2403.07153/) (2024)
- [Revisiting Multi-Scale Feature Fusion for Semantic Segmentation](https://mlsanity.com/paper/2203.12683/) (2022)

## Abstract (arXiv)

In deep CNN based models for semantic segmentation, high accuracy relies on rich spatial context (large receptive fields) and fine spatial details (high resolution), both of which incur high computational costs. In this paper, we propose a novel architecture that addresses both challenges and achieves state-of-the-art performance for semantic segmentation of high-resolution images and videos in real-time. The proposed architecture relies on our fast spatial attention, which is a simple yet efficient modification of the popular self-attention mechanism and captures the same rich spatial context at a small fraction of the computational cost, by changing the order of operations. Moreover, to efficiently process high-resolution input, we apply an additional spatial reduction to intermediate feature stages of the network with minimal loss in accuracy thanks to the use of the fast attention module to fuse features. We validate our method with a series of experiments, and show that results on multiple datasets demonstrate superior performance with better accuracy and speed compared to existing approaches for real-time semantic segmentation. On Cityscapes, our network achieves 74.4% mIoU at 72 FPS and 75.5% mIoU at 58 FPS on a single Titan X GPU, which is~~50% faster than the state-of-the-art while retaining the same accuracy.