mlsanityData to 14 Oct 2026

GLiT: Neural Architecture Search for Global and Local Image Transformer

arXiv 2107.02960 ↗First posted 7 Jul 2021cs.CVcited by 53

Not on the map. Why?

Abstract (arXiv)

We introduce the first Neural Architecture Search (NAS) method to find a better transformer architecture for image recognition. Recently, transformers without CNN-based backbones are found to achieve impressive performance for image recognition. However, the transformer is designed for NLP tasks and thus could be sub-optimal when directly used for image recognition. In order to improve the visual representation ability for transformers, we propose a new search space and searching algorithm. Specifically, we introduce a locality module that models the local correlations in images explicitly with fewer computational cost. With the locality module, our search space is defined to let the search algorithm freely trade off between global and local information as well as optimizing the low-level design choice in each module. To tackle the problem caused by huge search space, a hierarchical neural architecture search method is proposed to search the optimal vision transformer from two levels separately with the evolutionary algorithm. Extensive experiments on the ImageNet dataset demonstrate that our method can find more discriminative and efficient transformer variants than the ResNet family (e.g., ResNet101) and the baseline ViT for image classification.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Builds on 33 of its 38 references that are in the map. First posted on arXiv 7 Jul 2021.

Builds on

33 of 38 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2016Layer Normalizationcited by 5,669
  7. 2017Squeeze-and-Excitation Networkscited by 4,789
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  10. 2020Longformer: The Long-Document Transformercited by 3,413
  11. 2017Non-local Neural Networkscited by 2,293
  12. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
21 more
  1. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  2. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  3. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  4. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  5. 2020Pre-Trained Image Processing Transformercited by 893
  6. 2016Language Modeling with Gated Convolutional Networkscited by 702
  7. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  8. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  9. 2020Exploring Self-attention for Image Recognitioncited by 313
  10. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 235
  11. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 188
  12. 2017Neural Optimizer Search with Reinforcement Learningcited by 82
  13. 2020EcoNAS: Finding Proxies for Economical Neural Architecture Searchcited by 61
  14. 2019Recurrence is required to capture the representational dynamics of the human visual systemcited by 58
  15. 2019GradNet: Gradient-Guided Network for Visual Object Trackingcited by 41
  16. 2019Improving One-shot NAS by Suppressing the Posterior Fadingcited by 28
  17. 2020Evolving Search Space for Neural Architecture Searchcited by 23
  18. 2019Computation Reallocation for Object Detectioncited by 17
  19. 2020AutoTrans: Automating Transformer Design via Reinforced Architecture Searchcited by 12
  20. 2020Finding Fast Transformers: One-Shot Neural Architecture Search by Component Compositioncited by 10
  21. 2020Inception Convolution with Efficient Dilation Searchcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 18; 2023: 15; 2024: 9; 2025: 4; 2026: 2.

53 papers.

Sort:

By area:

  1. 2022Vision Transformer with Deformable Attentioncited by 295
  2. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 234
  3. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  4. 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 152
  5. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  6. 2022Backbone is All Your Need: A Simplified Architecture for Visual Object Trackingcited by 119
  7. 2022Inception Transformercited by 99
  8. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 80
  9. 2022Fast Vision Transformers with HiLo Attentioncited by 78
  10. 2021Global Vision Transformer Pruning with Hessian-Aware Saliencycited by 70
  11. 2022Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Spacecited by 66
  12. 2021An Image Patch is a Wave: Phase-Aware Vision MLPcited by 61
38 more
  1. 2023UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformerscited by 54
  2. 2022Neural Architecture Search for Spiking Neural Networkscited by 53
  3. 2022In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimationcited by 51
  4. 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformerscited by 46
  5. 2024A Survey on Transformer Compressioncited by 46
  6. 2023A Survey of Techniques for Optimizing Transformer Inferencecited by 35
  7. 2022Training-free Transformer Architecture Searchcited by 33
  8. 2022EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithmcited by 27
  9. 2024General surgery vision transformer: A video pre-trained foundation model for general surgerycited by 25
  10. 2021Pruning Self-attentions into Convolutional Layers in Single Pathcited by 24
  11. 2021BN-NAS: Neural Architecture Search with Batch Normalizationcited by 21
  12. 2023DAT++: Spatially Dynamic Vision Transformer with Deformable Attentioncited by 21
  13. 2023CrossFormer++: A Versatile Vision Transformer Hinging on Cross-scale Attentioncited by 18
  14. 2022Neural Architecture Search on Efficient Transformers and Beyondcited by 17
  15. 2021Mesa: A Memory-saving Training Framework for Transformerscited by 16
  16. 2022SepViT: Separable Vision Transformercited by 16
  17. 2022Auto-scaling Vision Transformers without Trainingcited by 15
  18. 2022RadFormer: Transformers with Global-Local Attention for Interpretable and Accurate Gallbladder Cancer Detectioncited by 12
  19. 2024A Timely Survey on Vision Transformer for Deepfake Detectioncited by 10
  20. 2023MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memorycited by 9
  21. 2022ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networkscited by 8
  22. 2024Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compressioncited by 8
  23. 2022Mixing and Shifting: Exploiting Global and Local Dependencies in Vision MLPscited by 6
  24. 2023TFormer: A Transmission-Friendly ViT Model for IoT Devicescited by 5
  25. 2024Lite-SAM Is Actually What You Need for Segment Everythingcited by 5
  26. 2023RingMo-lite: A Remote Sensing Multi-task Lightweight Network with CNN-Transformer Hybrid Frameworkcited by 4
  27. 2024DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisionscited by 4
  28. 2023Evolutionary Neural Architecture Search for Transformer in Knowledge Tracingcited by 3
  29. 2023FLORA: Fine-grained Low-Rank Architecture Search for Vision Transformercited by 3
  30. 2024Automatic Channel Pruning for Multi-Head Attentioncited by 3
  31. 2024HyTAS: A Hyperspectral Image Transformer Architecture Search Benchmark and Analysiscited by 3
  32. 2023Entropic Score metric: Decoupling Topology and Size in Training-free NAScited by 2
  33. 2024Neural Architecture Search based Global-local Vision Mamba for Palm-Vein Recognitioncited by 2
  34. 2023Neural Architecture Search: Two Constant Shared Weights Initialisationscited by 1
  35. 2025LightSNN: Lightweight Architecture Search for Sparse and Accurate Spiking Neural Networkscited by 1
  36. 2025Unmasking Facial DeepFakes: A Robust Multiview Detection Framework for Natural Imagescited by 1
  37. 2023Adaptively Clustering Neighbor Elements for Image-Text Generationcited by 0
  38. 2025Vision Transformer Neural Architecture Search for Out-of-Distribution Generalization: Benchmark and Insightscited by 0

50 most cited of 53 All 53 (JSON)