mlsanityData to 7 Oct 2026

Local-to-Global Self-Attention in Vision Transformers

arXiv 2107.04735 ↗First posted 10 Jul 2021cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Transformers have demonstrated great potential in computer vision tasks. To avoid dense computations of self-attentions in high-resolution visual data, some recent Transformer models adopt a hierarchical design, where self-attentions are only computed within local windows. This design significantly improves the efficiency but lacks global feature reasoning in early stages. In this work, we design a multi-path structure of the Transformer, which enables local-to-global reasoning at multiple granularities in each stage. The proposed framework is computationally efficient and highly effective. With a marginal increasement in computational overhead, our model achieves notable improvements in both image classification and semantic segmentation. Code is available at https://github.com/ljpadam/LG-Transformer

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 51 of its 57 references that are in the map. First posted on arXiv 10 Jul 2021.

Builds on

51 of 57 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2020End-to-End Object Detection with Transformerscited by 7,384
  11. 2016Densely Connected Convolutional Networkscited by 6,691
  12. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
38 more
  1. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  5. 2014Going Deeper with Convolutionscited by 4,231
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  7. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  9. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  10. 2016Pyramid Scene Parsing Networkcited by 2,472
  11. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  12. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  13. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  14. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  15. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  16. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  17. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  18. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  19. 2020Designing Network Design Spacescited by 954
  20. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  21. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  22. 2020Pre-Trained Image Processing Transformercited by 891
  23. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  24. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  25. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
  26. 2019Text Summarization with Pretrained Encoderscited by 553
  27. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  28. 2019Attention Augmented Convolutional Networkscited by 363
  29. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  30. 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 346
  31. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 333
  32. 2019NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detectioncited by 289
  33. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  34. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
  35. 2020Disentangled Non-Local Neural Networkscited by 132
  36. 2016A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detectioncited by 83
  37. 2017Residual Conv-Deconv Grid Network for Semantic Segmentationcited by 57
  38. 2019Adaptive Context Network for Scene Parsingcited by 51

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 7; 2023: 1; 2024: 0; 2025: 2.

13 papers.

Sort:

By area:

  1. 2022MixFormer: Mixing Features across Windows and Dimensionscited by 43
  2. 2021Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attentioncited by 36
  3. 2022When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention Mechanismcited by 30
  4. 2022Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasetscited by 28
  5. 2021ELSA: Enhanced Local Self-Attention for Vision Transformercited by 18
  6. 2021NomMer: Nominate Synergistic Context in Vision Transformer for Visual Recognitioncited by 12
  7. 2022Swinv2-Imagen: Hierarchical Vision Transformer Diffusion Models for Text-to-Image Generationcited by 7
  8. 2022Vision Transformers: From Semantic Segmentation to Dense Predictioncited by 6
  9. 2023Vision Backbone Enhancement via Multi-Stage Cross-Scale Attentioncited by 3
  10. 2022Improving Semantic Segmentation in Transformers using Hierarchical Inter-Level Attentioncited by 2
  11. 2025GasTwinFormer: A Hybrid Vision Transformer for Livestock Methane Emission Segmentation and Dietary Classification in Optical Gas Imagingcited by 1
  12. 2025Multi-label Classification with Panoptic Context Aggregation Networkscited by 1
1 more
  1. 2022Token Transformer: Can class token help window-based transformer build better long-range interactions?cited by 0