mlsanityData to 7 Oct 2026

A Review of Transformer-Based Models for Computer Vision Tasks: Capturing Global Context and Spatial Relationships

arXiv 2408.15178 ↗First posted 27 Aug 2024cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Transformer-based models have transformed the landscape of natural language processing (NLP) and are increasingly applied to computer vision tasks with remarkable success. These models, renowned for their ability to capture long-range dependencies and contextual information, offer a promising alternative to traditional convolutional neural networks (CNNs) in computer vision. In this review paper, we provide an extensive overview of various transformer architectures adapted for computer vision tasks. We delve into how these models capture global context and spatial relationships in images, empowering them to excel in tasks such as image classification, object detection, and segmentation. Analyzing the key components, training methodologies, and performance metrics of transformer-based models, we highlight their strengths, limitations, and recent advancements. Additionally, we discuss potential research directions and applications of transformer-based models in computer vision, offering insights into their implications for future advancements in the field.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 56 of its 130 references that are in the map. First posted on arXiv 27 Aug 2024.

Builds on

56 of 130 references.

References · 56

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2017Squeeze-and-Excitation Networkscited by 4,763
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  7. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  8. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  9. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  10. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  11. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  12. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
38 more
  1. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  2. 2022Vision Transformer with Deformable Attentioncited by 293
  3. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 286
  4. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  5. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  6. 2022Recurrent Vision Transformers for Object Detection with Event Camerascited by 158
  7. 2020Rethinking Transformer-based Set Prediction for Object Detectioncited by 156
  8. 2022DETRs with Hybrid Matchingcited by 149
  9. 2022ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyondcited by 117
  10. 2020End-to-End Object Detection with Adaptive Clustering Transformercited by 109
  11. 2022Global Context Vision Transformerscited by 98
  12. 2023Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformerscited by 91
  13. 2021Training Vision Transformers for Image Retrievalcited by 90
  14. 2021Contextual Transformer Networks for Visual Recognitioncited by 90
  15. 2021Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNetcited by 61
  16. 2022Patch-level Representation Learning for Self-supervised Vision Transformerscited by 60
  17. 2022AO2-DETR: Arbitrary-Oriented Object Detection Transformercited by 55
  18. 2023Rank-DETR for High Quality Object Detectioncited by 49
  19. 2021DPT: Deformable Patch-based Transformer for Visual Recognitioncited by 43
  20. 2021SOTR: Segmenting Objects with Transformerscited by 42
  21. 2021End-to-End Video Object Detection with Spatial-Temporal Transformerscited by 40
  22. 2021A Simple and Effective Positional Encoding for Transformerscited by 37
  23. 2021Rethinking Positional Encodingcited by 36
  24. 2021Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attentioncited by 36
  25. 2022Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detectioncited by 33
  26. 2023ARS-DETR: Aspect Ratio-Sensitive Detection Transformer for Aerial Oriented Object Detectioncited by 32
  27. 2023T-FFTRadNet: Object Detection with Swin Vision Transformers from Raw ADC Radar Signalscited by 24
  28. 2022HST: Hierarchical Swin Transformer for Compressed Image Super-resolutioncited by 20
  29. 2022Boosting vision transformers for image retrievalcited by 19
  30. 2022Enhanced Training of Query-Based Object Detection via Selective Query Recollectioncited by 17
  31. 2021T6D-Direct: Transformers for Multi-Object 6D Pose Direct Regressioncited by 16
  32. 2019Deep Object Co-segmentation via Spatial-Semantic Network Modulationcited by 15
  33. 2021Multi-Head Self-Attention via Vision Transformer for Zero-Shot Learningcited by 12
  34. 2021Investigating the Vision Transformer Model for Image Retrieval Taskscited by 11
  35. 2022HIPA: Hierarchical Patch Transformer for Single Image Super Resolutioncited by 11
  36. 2022Dynamic Sparse R-CNNcited by 11
  37. 2023DeFormer: Integrating Transformers with Deformable Models for 3D Shape Abstraction from a Single Imagecited by 11
  38. 2022Defect Transformer: An Efficient Hybrid Transformer Architecture for Surface Defect Detectioncited by 6

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 2.

11 papers.

  1. 2025PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinementcited by 2
  2. 2025Multi-label Classification with Panoptic Context Aggregation Networkscited by 1
  3. 2026DistillH-Mamba: A Hypergraph-Mamba-Based Knowledge Distillation Model for Efficient Impact Fall Detectioncited by 1
  4. 2025Unified Face Matching and Physical-Digital Spoofing Attack Detectioncited by 0
  5. 2025AttentionSmithy: A Modular Framework for Rapid Transformer Development and Customizationcited by 0
  6. 2025An Ensemble-Based Two-Step Framework for Classification of Pap Smear Cell Imagescited by 0
  7. 2025Exploring Kolmogorov-Arnold Network Expansions in Vision Transformers for Mitigating Catastrophic Forgetting in Continual Learningcited by 0
  8. 2025Region-Aware Deformable Convolutionscited by 0
  9. 2025Deep Learning Perspective of Scene Understanding in Autonomous Robotscited by 0
  10. 2025Learning Spatio-Temporal Feature Representations for Video-Based Gaze Estimationcited by 0
  11. 2026HiRo: A Compact Four-Directional Hierarchical Reservoir Token-Mixer for Efficient Image Classificationcited by 0