# Interlaced Sparse Self-Attention for Semantic Segmentation

arXiv: https://arxiv.org/abs/1907.12273 · HTML: https://mlsanity.com/paper/1907.12273/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2019.

## Built on by (most cited first, 30 of 53)

- [DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.14887/) (2021)
- [HRFormer: High-Resolution Transformer for Dense Prediction](https://mlsanity.com/paper/2110.09408/) (2021)
- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer](https://mlsanity.com/paper/2106.03650/) (2021)
- [An Attention Free Transformer](https://mlsanity.com/paper/2105.14103/) (2021)
- [Image Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2204.08610/) (2022)
- [On the Connection between Local Attention and Dynamic Depth-wise Convolution](https://mlsanity.com/paper/2106.04263/) (2021)
- [Capturing Omni-Range Context for Omnidirectional Segmentation](https://mlsanity.com/paper/2103.05687/) (2021)
- [Green Hierarchical Vision Transformer for Masked Image Modeling](https://mlsanity.com/paper/2205.13515/) (2022)
- [CTNet: Context-based Tandem Network for Semantic Segmentation](https://mlsanity.com/paper/2104.09805/) (2021)
- [ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer](https://mlsanity.com/paper/2203.10790/) (2022)
- [CDGNet: Class Distribution Guided Network for Human Parsing](https://mlsanity.com/paper/2111.14173/) (2021)
- [Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentation](https://mlsanity.com/paper/2304.13615/) (2023)
- [LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers](https://mlsanity.com/paper/2210.12755/) (2022)
- [SeaFormer++: Squeeze-enhanced Axial Transformer for Mobile Visual Recognition](https://mlsanity.com/paper/2301.13156/) (2023)
- [RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation](https://mlsanity.com/paper/2504.03166/) (2025)
- [BPKD: Boundary Privileged Knowledge Distillation For Semantic Segmentation](https://mlsanity.com/paper/2306.08075/) (2023)
- [FAKD: Feature Augmented Knowledge Distillation for Semantic Segmentation](https://mlsanity.com/paper/2208.14143/) (2022)
- [MCIBI++: Soft Mining Contextual Information Beyond Image for Semantic Segmentation](https://mlsanity.com/paper/2209.04471/) (2022)
- [FsaNet: Frequency Self-attention for Semantic Segmentation](https://mlsanity.com/paper/2211.15595/) (2022)
- [Compete to Win: Enhancing Pseudo Labels for Barely-supervised Medical Image Segmentation](https://mlsanity.com/paper/2304.07519/) (2023)
- [ORDNet: Capturing Omni-Range Dependencies for Scene Parsing](https://mlsanity.com/paper/2101.03929/) (2021)
- [IDRNet: Intervention-Driven Relation Network for Semantic Segmentation](https://mlsanity.com/paper/2310.10755/) (2023)
- [Retrieval and Localization with Observation Constraints](https://mlsanity.com/paper/2108.08516/) (2021)
- [PatchRefineNet: Improving Binary Segmentation by Incorporating Signals from Optimal Patch-wise Binarization](https://mlsanity.com/paper/2211.06560/) (2022)
- [Long-Range Grouping Transformer for Multi-View 3D Reconstruction](https://mlsanity.com/paper/2308.08724/) (2023)
- [Segment-Anything Models Achieve Zero-shot Robustness in Autonomous Driving](https://mlsanity.com/paper/2408.09839/) (2024)
- [UDTIRI: An Online Open-Source Intelligent Road Inspection Benchmark Suite](https://mlsanity.com/paper/2304.08842/) (2023)
- [SSSegmenation: An Open Source Supervised Semantic Segmentation Toolbox Based on PyTorch](https://mlsanity.com/paper/2305.17091/) (2023)
- [Pre-train, Adapt and Detect: Multi-Task Adapter Tuning for Camouflaged Object Detection](https://mlsanity.com/paper/2307.10685/) (2023)

## Abstract (arXiv)

In this paper, we present a so-called interlaced sparse self-attention approach to improve the efficiency of the self-attention mechanism for semantic segmentation. The main idea is that we factorize the dense affinity matrix as the product of two sparse affinity matrices. There are two successive attention modules each estimating a sparse affinity matrix. The first attention module is used to estimate the affinities within a subset of positions that have long spatial interval distances and the second attention module is used to estimate the affinities within a subset of positions that have short spatial interval distances. These two attention modules are designed so that each position is able to receive the information from all the other positions. In contrast to the original self-attention module, our approach decreases the computation and memory complexity substantially especially when processing high-resolution feature maps. We empirically verify the effectiveness of our approach on six challenging semantic segmentation benchmarks.