# Coordinate Attention for Efficient Mobile Network Design

arXiv: https://arxiv.org/abs/2103.02907 · HTML: https://mlsanity.com/paper/2103.02907/

## Key facts

- Cited by 241 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 52 references that are in the map.
- First posted on arXiv 4 Mar 2021.

## Builds on (most cited first, 30 of 44)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)

## Built on by (most cited first, 30 of 241)

- [DeepViT: Towards Deeper Vision Transformer](https://mlsanity.com/paper/2103.11886/) (2021)
- [Attention Mechanisms in Computer Vision: A Survey](https://mlsanity.com/paper/2111.07624/) (2021)
- [VOLO: Vision Outlooker for Visual Recognition](https://mlsanity.com/paper/2106.13112/) (2021)
- [DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation](https://mlsanity.com/paper/2106.06716/) (2021)
- [Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition](https://mlsanity.com/paper/2106.12368/) (2021)
- [Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition](https://mlsanity.com/paper/2106.15125/) (2021)
- [Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition](https://mlsanity.com/paper/2211.11943/) (2022)
- [Deep Model Reassembly](https://mlsanity.com/paper/2210.17409/) (2022)
- [MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities](https://mlsanity.com/paper/2205.00908/) (2022)
- [Multi-Task Dense Prediction via Mixture of Low-Rank Experts](https://mlsanity.com/paper/2403.17749/) (2024)
- [Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition](https://mlsanity.com/paper/2109.07270/) (2021)
- [CholecTriplet2021: A benchmark challenge for surgical action triplet recognition](https://mlsanity.com/paper/2204.04746/) (2022)
- [GhostNetV2: Enhance Cheap Operation with Long-Range Attention](https://mlsanity.com/paper/2211.12905/) (2022)
- [Weakly-Supervised Camouflaged Object Detection with Scribble Annotations](https://mlsanity.com/paper/2207.14083/) (2022)
- [Efficient Multi-Scale Attention Module with Cross-Spatial Learning](https://mlsanity.com/paper/2305.13563/) (2023)
- [NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results](https://mlsanity.com/paper/2106.01439/) (2021)
- [CoNIC Challenge: Pushing the Frontiers of Nuclear Detection, Segmentation, Classification and Counting](https://mlsanity.com/paper/2303.06274/) (2023)
- [Feature Distillation Interaction Weighting Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2112.08655/) (2021)
- [HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection](https://mlsanity.com/paper/2301.03036/) (2023)
- [LR-FPN: Enhancing Remote Sensing Object Detection with Location Refined Feature Pyramid Network](https://mlsanity.com/paper/2404.01614/) (2024)
- [Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentation](https://mlsanity.com/paper/2405.06228/) (2024)
- [SeaFormer++: Squeeze-enhanced Axial Transformer for Mobile Visual Recognition](https://mlsanity.com/paper/2301.13156/) (2023)
- [FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation](https://mlsanity.com/paper/2109.00699/) (2021)
- [AsymFormer: Asymmetrical Cross-Modal Representation Learning for Mobile Platform Real-Time RGB-D Semantic Segmentation](https://mlsanity.com/paper/2309.14065/) (2023)
- [Effective Image Tampering Localization via Enhanced Transformer and Co-attention Fusion](https://mlsanity.com/paper/2309.09306/) (2023)
- [Benchmarking and scaling of deep learning models for land cover image classification](https://mlsanity.com/paper/2111.09451/) (2021)
- [MSDet: Receptive Field Enhanced Multiscale Detection for Tiny Pulmonary Nodule](https://mlsanity.com/paper/2409.14028/) (2024)
- [Strip R-CNN: Large Strip Convolution for Remote Sensing Object Detection](https://mlsanity.com/paper/2501.03775/) (2025)
- [Efficient Mirror Detection via Multi-level Heterogeneous Learning](https://mlsanity.com/paper/2211.15644/) (2022)
- [Rotate to Scan: UNet-like Mamba with Triplet SSM Module for Medical Image Segmentation](https://mlsanity.com/paper/2403.17701/) (2024)

## Abstract (arXiv)

Recent studies on mobile network design have demonstrated the remarkable effectiveness of channel attention (e.g., the Squeeze-and-Excitation attention) for lifting model performance, but they generally neglect the positional information, which is important for generating spatially selective attention maps. In this paper, we propose a novel attention mechanism for mobile networks by embedding positional information into channel attention, which we call "coordinate attention". Unlike channel attention that transforms a feature tensor to a single feature vector via 2D global pooling, the coordinate attention factorizes channel attention into two 1D feature encoding processes that aggregate features along the two spatial directions, respectively. In this way, long-range dependencies can be captured along one spatial direction and meanwhile precise positional information can be preserved along the other spatial direction. The resulting feature maps are then encoded separately into a pair of direction-aware and position-sensitive attention maps that can be complementarily applied to the input feature map to augment the representations of the objects of interest. Our coordinate attention is simple and can be flexibly plugged into classic mobile networks, such as MobileNetV2, MobileNeXt, and EfficientNet with nearly no computational overhead. Extensive experiments demonstrate that our coordinate attention is not only beneficial to ImageNet classification but more interestingly, behaves better in down-stream tasks, such as object detection and semantic segmentation. Code is available at https://github.com/Andrew-Qibin/CoordAttention.