# Lawin Transformer: Improving Semantic Segmentation Transformer with Multi-Scale Representations via Large Window Attention

arXiv: https://arxiv.org/abs/2201.01615 · HTML: https://mlsanity.com/paper/2201.01615/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 54 references that are in the map.
- First posted on arXiv 5 Jan 2022.

## Builds on (most cited first, 30 of 43)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)

## Built on by (most cited first, 30 of 30)

- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [Transformer Scale Gate for Semantic Segmentation](https://mlsanity.com/paper/2205.07056/) (2022)
- [Lightweight Real-time Semantic Segmentation Network with Efficient Transformer and CNN](https://mlsanity.com/paper/2302.10484/) (2023)
- [Disentangle then Parse:Night-time Semantic Segmentation with Illumination Disentanglement](https://mlsanity.com/paper/2307.09362/) (2023)
- [AI and 6G into the Metaverse: Fundamentals, Challenges and Future Research Trends](https://mlsanity.com/paper/2208.10921/) (2022)
- [SERNet-Former: Semantic Segmentation by Efficient Residual Network with Attention-Boosting Gates and Attention-Fusion Networks](https://mlsanity.com/paper/2401.15741/) (2024)
- [Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning](https://mlsanity.com/paper/2203.10395/) (2022)
- [HarDNet-DFUS: An Enhanced Harmonically-Connected Network for Diabetic Foot Ulcer Image Segmentation and Colonoscopy Polyp Segmentation](https://mlsanity.com/paper/2209.07313/) (2022)
- [Improving Normalization with the James-Stein Estimator](https://mlsanity.com/paper/2312.00313/) (2023)
- [UP-CrackNet: Unsupervised Pixel-Wise Road Crack Detection via Adversarial Image Restoration](https://mlsanity.com/paper/2401.15647/) (2024)
- [HAFormer: Unleashing the Power of Hierarchy-Aware Features for Lightweight Semantic Segmentation](https://mlsanity.com/paper/2407.07441/) (2024)
- [Representation Separation for Semantic Segmentation with Vision Transformers](https://mlsanity.com/paper/2212.13764/) (2022)
- [Transferable Attack for Semantic Segmentation](https://mlsanity.com/paper/2307.16572/) (2023)
- [Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights](https://mlsanity.com/paper/2403.19882/) (2024)
- [Probing Contextual Diversity for Dense Out-of-Distribution Detection](https://mlsanity.com/paper/2208.14195/) (2022)
- [Minimalist and High-Performance Semantic Segmentation with Plain Vision Transformers](https://mlsanity.com/paper/2310.12755/) (2023)
- [Real Time Egocentric Segmentation for Video-self Avatar in Mixed Reality](https://mlsanity.com/paper/2207.01296/) (2022)
- [Mutual Information Learned Classifiers: an Information-theoretic Viewpoint of Training Deep Learning Classification Systems](https://mlsanity.com/paper/2209.10058/) (2022)
- [Cross-level Attention with Overlapped Windows for Camouflaged Object Detection](https://mlsanity.com/paper/2311.16618/) (2023)
- [StarCraftImage: A Dataset For Prototyping Spatial Reasoning Methods For Multi-Agent Environments](https://mlsanity.com/paper/2401.04290/) (2024)
- [MarsSeg: Mars Surface Semantic Segmentation with Multi-level Extractor and Connector](https://mlsanity.com/paper/2404.04155/) (2024)
- [DocParseNet: Advanced Semantic Segmentation and OCR Embeddings for Efficient Scanned Document Annotation](https://mlsanity.com/paper/2406.17591/) (2024)
- [Borrowing from yourself: Faster future video segmentation with partial channel update](https://mlsanity.com/paper/2202.05748/) (2022)
- [Mutual Information Learned Classifiers: an Information-theoretic Viewpoint of Training Deep Learning Classification Systems](https://mlsanity.com/paper/2210.01000/) (2022)
- [Diabetic Foot Ulcer Grand Challenge 2022 Summary](https://mlsanity.com/paper/2304.12001/) (2023)
- [P2AT: Pyramid Pooling Axial Transformer for Real-time Semantic Segmentation](https://mlsanity.com/paper/2310.15025/) (2023)
- [A Decoding Scheme with Successive Aggregation of Multi-Level Features for Light-Weight Semantic Segmentation](https://mlsanity.com/paper/2402.11201/) (2024)
- [Colorectal Cancer Tumor Grade Segmentation in Digital Histopathology Images: From Giga to Mini Challenge](https://mlsanity.com/paper/2507.04681/) (2025)
- [APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms](https://mlsanity.com/paper/2606.30577/) (2026)

## Abstract (arXiv)

Multi-scale representations are crucial for semantic segmentation. The community has witnessed the flourish of semantic segmentation convolutional neural networks (CNN) exploiting multi-scale contextual information. Motivated by that the vision transformer (ViT) is powerful in image classification, some semantic segmentation ViTs are recently proposed, most of them attaining impressive results but at a cost of computational economy. In this paper, we succeed in introducing multi-scale representations into semantic segmentation ViT via window attention mechanism and further improves the performance and efficiency. To this end, we introduce large window attention which allows the local window to query a larger area of context window at only a little computation overhead. By regulating the ratio of the context area to the query area, we enable the textitlarge window attention to capture the contextual information at multiple scales. Moreover, the framework of spatial pyramid pooling is adopted to collaborate with textitthe large window attention, which presents a novel decoder named large window attention spatial pyramid pooling (LawinASPP) for semantic segmentation ViT. Our resulting ViT, Lawin Transformer, is composed of an efficient hierachical vision transformer (HVT) as encoder and a LawinASPP as decoder. The empirical results demonstrate that Lawin Transformer offers an improved efficiency compared to the existing method. Lawin Transformer further sets new state-of-the-art performance on Cityscapes (84.4% mIoU), ADE20K (56.2% mIoU) and COCO-Stuff datasets. The code will be released at https://github.com/yan-hao-tian/lawin