mlsanityData to 7 Oct 2026

Visual Transformers: Token-based Image Representation and Processing for Computer Vision

arXiv 2006.03677 ↗First posted 5 Jun 2020cs.CV cs.LG eess.IVcited by 333

Not on the map. Why?

Abstract (arXiv)

Computer vision has achieved remarkable success by (a) representing images as uniformly-arranged pixel arrays and (b) convolving highly-localized features. However, convolutions treat all image pixels equally regardless of importance; explicitly model all concepts across all images, regardless of content; and struggle to relate spatially-distant concepts. In this work, we challenge this paradigm by (a) representing images as semantic visual tokens and (b) running transformers to densely model token relationships. Critically, our Visual Transformer operates in a semantic token space, judiciously attending to different image parts based on context. This is in sharp contrast to pixel-space transformers that require orders-of-magnitude more compute. Using an advanced training recipe, our VTs significantly outperform their convolutional counterparts, raising ResNet accuracy on ImageNet top-1 by 4.6 to 7 points while using fewer FLOPs and parameters. For semantic segmentation on LIP and COCO-stuff, VT-based feature pyramid networks (FPN) achieve 0.35 points higher mIoU while reducing the FPN module's FLOPs by 6.5x.

Key facts. Cited by 333 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 67; 2022: 51; 2023: 43; 2024: 77; 2025: 57; 2026: 38.

333 papers.

Sort:

By area:

  1. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  2. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  3. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  4. 2021Multiscale Vision Transformerscited by 896
  5. 2021Going deeper with Image Transformerscited by 759
  6. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  7. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  8. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  9. 2022Activating More Pixels in Image Super-Resolution Transformercited by 433
  10. 2021Remote Sensing Image Change Detection with Transformerscited by 305
  11. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  12. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
38 more
  1. 2021StyTr2: Image Style Transfer with Transformerscited by 186
  2. 2022Recurrent Video Restoration Transformer with Guided Deformable Attentioncited by 168
  3. 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruningcited by 165
  4. 2021Conformer: Local Features Coupling Global Representations for Visual Recognitioncited by 164
  5. 2021VOLO: Vision Outlooker for Visual Recognitioncited by 164
  6. 2022Efficient Long-Range Attention Network for Image Super-resolutioncited by 151
  7. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  8. 2021TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localizationcited by 122
  9. 2021A Survey of Visual Transformerscited by 117
  10. 2021Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformercited by 116
  11. 2021Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstructioncited by 110
  12. 2021LambdaNetworks: Modeling Long-Range Interactions Without Attentioncited by 107
  13. 2022CenterCLIP: Token Clustering for Efficient Text-Video Retrievalcited by 86
  14. 2021OadTR: Online Action Detection with Transformerscited by 83
  15. 2025PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slidescited by 83
  16. 2021Causal Attention for Unbiased Visual Recognitioncited by 80
  17. 2024AutoFlow: Automated Workflow Generation for Large Language Model Agentscited by 76
  18. 2022MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstructioncited by 72
  19. 2024Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategycited by 71
  20. 2021Token Pooling in Vision Transformerscited by 68
  21. 2022Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstructioncited by 67
  22. 2022Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Spacecited by 66
  23. 2023HAT: Hybrid Attention Transformer for Image Restorationcited by 64
  24. 2024Anisotropy Is Inherent to Self-Attention in Transformerscited by 62
  25. 2021TransCrowd: weakly-supervised crowd counting with transformerscited by 61
  26. 2021An Image Patch is a Wave: Phase-Aware Vision MLPcited by 61
  27. 2021Sub-word Level Lip Reading With Visual Attentioncited by 60
  28. 2024R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Modelcited by 56
  29. 2021GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformercited by 54
  30. 2021AAformer: Auto-Aligned Transformer for Person Re-Identificationcited by 48
  31. 2022MatteFormer: Transformer-Based Image Matting via Prior-Tokenscited by 48
  32. 2021TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Networkcited by 45
  33. 2022Region-Aware Face Swappingcited by 45
  34. 2021Vision Transformer with Progressive Samplingcited by 44
  35. 2022Flow-Guided Sparse Transformer for Video Deblurringcited by 43
  36. 2025CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AIcited by 42
  37. 2022ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Designcited by 41
  38. 2021Bridging Global Context Interactions for High-Fidelity Image Completioncited by 40

50 most cited of 333 All 333 (JSON)