# Efficient Multi-Scale Attention Module with Cross-Spatial Learning

arXiv: https://arxiv.org/abs/2305.13563 · HTML: https://mlsanity.com/paper/2305.13563/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 28 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Coordinate Attention for Efficient Mobile Network Design](https://mlsanity.com/paper/2103.02907/) (2021)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [A2-Nets: Double Attention Networks](https://mlsanity.com/paper/1810.11579/) (2018)
- [TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios](https://mlsanity.com/paper/2108.11539/) (2021)
- [Rotate to Attend: Convolutional Triplet Attention Module](https://mlsanity.com/paper/2010.03045/) (2020)
- [SA-Net: Shuffle Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/2102.00240/) (2021)
- [Polarized Self-Attention: Towards High-quality Pixel-wise Regression](https://mlsanity.com/paper/2107.00782/) (2021)
- [Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks](https://mlsanity.com/paper/1905.09646/) (2019)
- [MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning](https://mlsanity.com/paper/1911.09483/) (2019)
- [Non-deep Networks](https://mlsanity.com/paper/2110.07641/) (2021)
- [NAM: Normalization-based Attention Module](https://mlsanity.com/paper/2111.12419/) (2021)

## Built on by (most cited first, 30 of 41)

- [MobileIE: An Extremely Lightweight and Effective ConvNet for Real-Time Image Enhancement on Mobile Devices](https://mlsanity.com/paper/2507.01838/) (2025)
- [SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention](https://mlsanity.com/paper/2407.05128/) (2024)
- [Enhanced Self-Checkout System for Retail Based on Improved YOLOv10](https://mlsanity.com/paper/2407.21308/) (2024)
- [Attention-guided Feature Distillation for Semantic Segmentation](https://mlsanity.com/paper/2403.05451/) (2024)
- [SOD-YOLOv8 -- Enhancing YOLOv8 for Small Object Detection in Traffic Scenes](https://mlsanity.com/paper/2408.04786/) (2024)
- [Advancing Autonomous Vehicle Intelligence: Deep Learning and Multimodal LLM for Traffic Sign Recognition and Robust Lane Detection](https://mlsanity.com/paper/2503.06313/) (2025)
- [Enhancing Layer Attention Efficiency through Pruning Redundant Retrievals](https://mlsanity.com/paper/2503.06473/) (2025)
- [An Efficient Aerial Image Detection with Variable Receptive Fields](https://mlsanity.com/paper/2504.15165/) (2025)
- [MRC-DETR: An Adaptive Multi-Residual Coupled Transformer for Bare Board PCB Defect Detection](https://mlsanity.com/paper/2507.03386/) (2025)
- [EMDFNet: Efficient Multi-scale and Diverse Feature Network for Traffic Sign Detection](https://mlsanity.com/paper/2408.14189/) (2024)
- [MASF-YOLO: An Improved YOLOv11 Network for Small Object Detection on Drone View](https://mlsanity.com/paper/2504.18136/) (2025)
- [EndoARSS: Adapting Spatially-Aware Foundation Model for Efficient Activity Recognition and Semantic Segmentation in Endoscopic Surgery](https://mlsanity.com/paper/2506.06830/) (2025)
- [Geo-RepNet: Geometry-Aware Representation Learning for Surgical Phase Recognition in Endoscopic Submucosal Dissection](https://mlsanity.com/paper/2507.09294/) (2025)
- [SDE-DET: A Precision Network for Shatian Pomelo Detection in Complex Orchard Environments](https://mlsanity.com/paper/2509.19990/) (2025)
- [XYZCylinder: Towards Compatible Feed-Forward 3D Gaussian Splatting for Driving Scenes via Unified Cylinder Lifting Method](https://mlsanity.com/paper/2510.07856/) (2025)
- [Fused attention mechanism-based ore sorting network](https://mlsanity.com/paper/2405.02785/) (2024)
- [RemoCap: Disentangled Representation Learning for Motion Capture](https://mlsanity.com/paper/2405.12724/) (2024)
- [Greit-HRNet: Grouped Lightweight High-Resolution Network for Human Pose Estimation](https://mlsanity.com/paper/2407.07389/) (2024)
- [A lightweight model FDM-YOLO for small target improvement based on YOLOv8](https://mlsanity.com/paper/2503.04452/) (2025)
- [KunlunBaize: LLM with Multi-Scale Convolution and Multi-Token Prediction Under TransformerX Framework](https://mlsanity.com/paper/2503.04784/) (2025)
- [OminiAdapt: Learning Cross-Task Invariance for Robust and Environment-Aware Robotic Manipulation](https://mlsanity.com/paper/2503.21257/) (2025)
- [SMDDFNet: State-space Modeling and Dynamic Dual Fusion Network for Traffic Sign Detection](https://mlsanity.com/paper/2505.05491/) (2025)
- [A High-Performance Thermal Infrared Object Detection Framework with Centralized Regulation](https://mlsanity.com/paper/2505.10825/) (2025)
- [J-DDL: Surface Damage Detection and Localization System for Fighter Aircraft](https://mlsanity.com/paper/2506.10505/) (2025)
- [DGE-YOLO: Dual-Branch Gathering and Attention for Accurate UAV Object Detection](https://mlsanity.com/paper/2506.23252/) (2025)
- [Efficient SAR Vessel Detection for FPGA-Based On-Satellite Sensing](https://mlsanity.com/paper/2507.04842/) (2025)
- [HGNet: High-Order Spatial Awareness Hypergraph and Multi-Scale Context Attention Network for Colorectal Polyp Detection](https://mlsanity.com/paper/2507.04880/) (2025)
- [A Two-Stage Strategy for Mitosis Detection Using Improved YOLO11x Proposals and ConvNeXt Classification](https://mlsanity.com/paper/2509.02627/) (2025)
- [Vi-SAFE: A Spatial-Temporal Framework for Efficient Violence Detection in Public Surveillance](https://mlsanity.com/paper/2509.13210/) (2025)
- [YOLOv11-Litchi: Efficient Litchi Fruit Detection based on UAV-Captured Agricultural Imagery in Complex Orchard Environments](https://mlsanity.com/paper/2510.10141/) (2025)

## Abstract (arXiv)

Remarkable effectiveness of the channel or spatial attention mechanisms for producing more discernible feature representation are illustrated in various computer vision tasks. However, modeling the cross-channel relationships with channel dimensionality reduction may bring side effect in extracting deep visual representations. In this paper, a novel efficient multi-scale attention (EMA) module is proposed. Focusing on retaining the information on per channel and decreasing the computational overhead, we reshape the partly channels into the batch dimensions and group the channel dimensions into multiple sub-features which make the spatial semantic features well-distributed inside each feature group. Specifically, apart from encoding the global information to re-calibrate the channel-wise weight in each parallel branch, the output features of the two parallel branches are further aggregated by a cross-dimension interaction for capturing pixel-level pairwise relationship. We conduct extensive ablation studies and experiments on image classification and object detection tasks with popular benchmarks (e.g., CIFAR-100, ImageNet-1k, MS COCO and VisDrone2019) for evaluating its performance.