# Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions

arXiv: https://arxiv.org/abs/2112.05561 · HTML: https://mlsanity.com/paper/2112.05561/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [BAM: Bottleneck Attention Module](https://mlsanity.com/paper/1807.06514/) (2018)
- [Attention Branch Network: Learning of Attention Mechanism for Visual Explanation](https://mlsanity.com/paper/1812.10025/) (2018)
- [Rotate to Attend: Convolutional Triplet Attention Module](https://mlsanity.com/paper/2010.03045/) (2020)

## Built on by (most cited first, 30 of 39)

- [STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolution](https://mlsanity.com/paper/2501.02976/) (2025)
- [DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens](https://mlsanity.com/paper/2502.11493/) (2025)
- [Fracture Detection in Pediatric Wrist Trauma X-ray Images Using YOLOv8 Algorithm](https://mlsanity.com/paper/2304.05071/) (2023)
- [RFAConv: Receptive-Field Attention Convolution for Improving Convolutional Neural Networks](https://mlsanity.com/paper/2304.03198/) (2023)
- [YOLOv9 for Fracture Detection in Pediatric Wrist Trauma X-ray Images](https://mlsanity.com/paper/2403.11249/) (2024)
- [BRAU-Net++: U-Shaped Hybrid CNN-Transformer Network for Medical Image Segmentation](https://mlsanity.com/paper/2401.00722/) (2024)
- [YOLOv8-AM: YOLOv8 Based on Effective Attention Mechanisms for Pediatric Wrist Fracture Detection](https://mlsanity.com/paper/2402.09329/) (2024)
- [LoCI-DiffCom: Longitudinal Consistency-Informed Diffusion Model for 3D Infant Brain Image Completion](https://mlsanity.com/paper/2405.10691/) (2024)
- [High Frequency Matters: Uncertainty Guided Image Compression with Wavelet Diffusion](https://mlsanity.com/paper/2407.12538/) (2024)
- [DAM: Dynamic Attention Mask for Long-Context Large Language Model Inference Acceleration](https://mlsanity.com/paper/2506.11104/) (2025)
- [RaLiFlow: Scene Flow Estimation with 4D Radar and LiDAR Point Clouds](https://mlsanity.com/paper/2512.10376/) (2025)
- [DeepLOC: Deep Learning-based Bone Pathology Localization and Classification in Wrist X-ray Images](https://mlsanity.com/paper/2308.12727/) (2023)
- [Underwater target detection based on improved YOLOv7](https://mlsanity.com/paper/2302.06939/) (2023)
- [Dynamic Graph Convolutional Network with Attention Fusion for Traffic Flow Prediction](https://mlsanity.com/paper/2302.12598/) (2023)
- [YOLO-TLA: An Efficient and Lightweight Small Object Detection Model based on YOLOv5](https://mlsanity.com/paper/2402.14309/) (2024)
- [Global Context Modeling in YOLOv8 for Pediatric Wrist Fracture Detection](https://mlsanity.com/paper/2407.03163/) (2024)
- [Pediatric Wrist Fracture Detection Using Feature Context Excitation Modules in X-ray Images](https://mlsanity.com/paper/2410.01031/) (2024)
- [Cott-ADNet: Lightweight Real-Time Cotton Boll and Flower Detection Under Field Conditions](https://mlsanity.com/paper/2509.12442/) (2025)
- [Semantic Interleaving Global Channel Attention for Multilabel Remote Sensing Image Classification](https://mlsanity.com/paper/2208.02613/) (2022)
- [Light-YOLOv5: A Lightweight Algorithm for Improved YOLOv5 in Complex Fire Scenarios](https://mlsanity.com/paper/2208.13422/) (2022)
- [Finger Multimodal Feature Fusion and Recognition Based on Channel Spatial Attention](https://mlsanity.com/paper/2209.02368/) (2022)
- [Two-Stage Adaptive Network for Semi-Supervised Cross-Domain Crater Detection under Varying Scenario Distributions](https://mlsanity.com/paper/2312.06169/) (2023)
- [EncodeNet: A Framework for Boosting DNN Accuracy with Entropy-driven Generalized Converting Autoencoder](https://mlsanity.com/paper/2404.13770/) (2024)
- [GAN-HA: A generative adversarial network with a novel heterogeneous dual-discriminator network and a new attention-based fusion strategy for infrared and visible image fusion](https://mlsanity.com/paper/2404.15992/) (2024)
- [GSO-YOLO: Global Stability Optimization YOLO for Construction Site Detection](https://mlsanity.com/paper/2407.00906/) (2024)
- [Bi-temporal Gaussian Feature Dependency Guided Change Detection in Remote Sensing Images](https://mlsanity.com/paper/2410.09539/) (2024)
- [Improving Generalization Performance of YOLOv8 for Camera Trap Object Detection](https://mlsanity.com/paper/2412.14211/) (2024)
- [Improved YOLOv7x-Based Defect Detection Algorithm for Power Equipment](https://mlsanity.com/paper/2502.17961/) (2025)
- [DMAGaze: Gaze Estimation Based on Feature Disentanglement and Multi-Scale Attention](https://mlsanity.com/paper/2504.11160/) (2025)
- [Document Image Rectification Bases on Self-Adaptive Multitask Fusion](https://mlsanity.com/paper/2505.06038/) (2025)

## Abstract (arXiv)

A variety of attention mechanisms have been studied to improve the performance of various computer vision tasks. However, the prior methods overlooked the significance of retaining the information on both channel and spatial aspects to enhance the cross-dimension interactions. Therefore, we propose a global attention mechanism that boosts the performance of deep neural networks by reducing information reduction and magnifying the global interactive representations. We introduce 3D-permutation with multilayer-perceptron for channel attention alongside a convolutional spatial attention submodule. The evaluation of the proposed mechanism for the image classification task on CIFAR-100 and ImageNet-1K indicates that our method stably outperforms several recent attention mechanisms with both ResNet and lightweight MobileNet.