# GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond

arXiv: https://arxiv.org/abs/1904.11492 · HTML: https://mlsanity.com/paper/1904.11492/

## Key facts

- Cited by 331 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2019.

## Built on by (most cited first, 30 of 331)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Bottleneck Transformers for Visual Recognition](https://mlsanity.com/paper/2101.11605/) (2021)
- [Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2201.00520/) (2022)
- [Focal Self-attention for Local-Global Interactions in Vision Transformers](https://mlsanity.com/paper/2107.00641/) (2021)
- [Coordinate Attention for Efficient Mobile Network Design](https://mlsanity.com/paper/2103.02907/) (2021)
- [Attention Mechanisms in Computer Vision: A Survey](https://mlsanity.com/paper/2111.07624/) (2021)
- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [Focal Modulation Networks](https://mlsanity.com/paper/2203.11926/) (2022)
- [You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection](https://mlsanity.com/paper/2106.00666/) (2021)
- [Conformer: Local Features Coupling Global Representations for Visual Recognition](https://mlsanity.com/paper/2105.03889/) (2021)
- [Focal and Global Knowledge Distillation for Detectors](https://mlsanity.com/paper/2111.11837/) (2021)
- [Visformer: The Vision-friendly Transformer](https://mlsanity.com/paper/2104.12533/) (2021)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization](https://mlsanity.com/paper/2103.14862/) (2021)
- [A Survey of Visual Transformers](https://mlsanity.com/paper/2111.06091/) (2021)
- [Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks](https://mlsanity.com/paper/2105.02358/) (2021)
- [Involution: Inverting the Inherence of Convolution for Visual Recognition](https://mlsanity.com/paper/2103.06255/) (2021)
- [Large Selective Kernel Network for Remote Sensing Object Detection](https://mlsanity.com/paper/2303.09030/) (2023)
- [Image Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2204.08610/) (2022)
- [Self-Support Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2207.11549/) (2022)
- [Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion](https://mlsanity.com/paper/2202.10304/) (2022)
- [Contextual Transformer Networks for Visual Recognition](https://mlsanity.com/paper/2107.12292/) (2021)
- [You Only Learn One Representation: Unified Network for Multiple Tasks](https://mlsanity.com/paper/2105.04206/) (2021)
- [On the Integration of Self-Attention and Convolution](https://mlsanity.com/paper/2111.14556/) (2021)
- [Hire-MLP: Vision MLP via Hierarchical Rearrangement](https://mlsanity.com/paper/2108.13341/) (2021)

## Abstract (arXiv)

The Non-Local Network (NLNet) presents a pioneering approach for capturing long-range dependencies, via aggregating query-specific global context to each query position. However, through a rigorous empirical analysis, we have found that the global contexts modeled by non-local network are almost the same for different query positions within an image. In this paper, we take advantage of this finding to create a simplified network based on a query-independent formulation, which maintains the accuracy of NLNet but with significantly less computation. We further observe that this simplified design shares similar structure with Squeeze-Excitation Network (SENet). Hence we unify them into a three-step general framework for global context modeling. Within the general framework, we design a better instantiation, called the global context (GC) block, which is lightweight and can effectively model the global context. The lightweight property allows us to apply it for multiple layers in a backbone network to construct a global context network (GCNet), which generally outperforms both simplified NLNet and SENet on major benchmarks for various recognition tasks. The code and configurations are released at https://github.com/xvjiarui/GCNet.