mlsanityData to 7 Oct 2026

Vision Transformers Need More Than Registers

arXiv 2602.22394 ↗First posted 25 Feb 2026cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs), when pre-trained on large-scale data, provide general-purpose representations for diverse downstream tasks. However, artifacts in ViTs are widely observed across different supervision paradigms and downstream tasks. Through systematic analysis of artifacts in ViTs, we find that their fundamental mechanisms have yet to be sufficiently elucidated. In this paper, through systematic analysis, we conclude that these artifacts originate from a lazy aggregation behavior: ViT uses semantically irrelevant background patches as shortcuts to represent global semantics, driven by global attention and Coarse-grained semantic supervision. Our solution selectively integrates patch features into the CLS token, reducing the influence of background-dominated shortcuts and consistently improving performance across 12 benchmarks under label-, text-, and self-supervision. We hope this work offers a new perspective on ViT behavior.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 50 of its 60 references that are in the map. First posted on arXiv 25 Feb 2026.

Builds on

50 of 60 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2017Mask R-CNNcited by 7,259
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  10. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  12. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
38 more
  1. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  2. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  3. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  4. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  5. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  6. 2023Vision Transformers Need Registerscited by 844
  7. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  8. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  9. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  10. 2021Extract Free Dense Labels from CLIPcited by 580
  11. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  12. 2023Demystifying CLIP Datacited by 265
  13. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
  14. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  15. 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 169
  16. 2022F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Modelscited by 149
  17. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 145
  18. 2023CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matchingcited by 133
  19. 2023Aligning Bag of Regions for Open-Vocabulary Object Detectioncited by 119
  20. 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Predictioncited by 114
  21. 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 110
  22. 2022Learning Object-Language Alignments for Open-Vocabulary Object Detectioncited by 95
  23. 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 94
  24. 2023Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformerscited by 91
  25. 2024Pay Attention to Your Neighbours: Training-Free Open-Vocabulary Semantic Segmentationcited by 76
  26. 2023EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignmentcited by 44
  27. 2024Mamba-R: Vision Mamba ALSO Needs Registerscited by 41
  28. 2025Vision Transformers Don't Need Trained Registerscited by 39
  29. 2024SegMAN: Omni-scale Context Modeling with State Space Models and Local Attention for Semantic Segmentationcited by 31
  30. 2022Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learningcited by 27
  31. 2025OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernelscited by 24
  32. 2023Emergence of Segmentation with Minimalistic White-Box Transformerscited by 23
  33. 2023TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognitioncited by 21
  34. 2023Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit?cited by 16
  35. 2024SparX: A Sparse Cross-Layer Connection Mechanism for Hierarchical Vision Mamba and Transformer Networkscited by 14
  36. 2025Rethinking Query-based Transformer for Continual Image Segmentationcited by 14
  37. 2025Vision Transformers with Self-Distilled Registerscited by 12
  38. 2024The devil is in the object boundary: towards annotation-free instance segmentation using Foundation Modelscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026VGGT-Omegacited by 20
  2. 2026Registers Matter for Pixel-Space Diffusion Transformerscited by 2
  3. 2026LookWhen? Fast Video Recognition by Learning When, Where, and What to Computecited by 1
  4. 2026Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Modelscited by 1
  5. 2026NTR: Neural Token Reconstruction for Scene Token Bottleneck in End-to-End Drivingcited by 1
  6. 2026Zero-Ablation Overstates Register Content Dependence in DINO Vision Transformerscited by 0
  7. 2026SARA: Semantically Adaptive Relational Alignment for Video Diffusion Modelscited by 0
  8. 2026What-Where Transformer: A Slot-Centric Visual Backbone for Concurrent Representation and Localizationcited by 0
  9. 2026When Dynamics Models Read the Wrong Time Steps: Label-Free Event Credit Re-Anchoring for Robust Global Readoutscited by 0
  10. 2026More Structure, Not More Capacity: Object-Centric Representations for Visuomotor Imitation Learningcited by 0
  11. 2026Feed-Forward Hierarchical Gaussian Diffusion for Extreme CT Reconstructioncited by 0
  12. 2026Beyond Classification: Structured Supervision Aligns Visual Evidence with Medical Semanticscited by 0
2 more
  1. 2026CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Frameworkcited by 0
  2. 2026OFBD: Object-Focused Background Debiasing for Long-Tailed Learningcited by 0