mlsanityData to 7 Oct 2026

Finding Distributed Object-Centric Properties in Self-Supervised Transformers

arXiv 2603.26127 ↗First posted 27 Mar 2026cs.CV cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Self-supervised Vision Transformers (ViTs) like DINO show an emergent ability to discover objects, typically observed in [CLS] token attention maps of the final layer. However, these maps often contain spurious activations resulting in poor localization of objects. This is because the [CLS] token, trained on an image-level objective, summarizes the entire image instead of focusing on objects. This aggregation dilutes the object-centric information existing in the local, patch-level interactions. We analyze this by computing inter-patch similarity using patch-level attention components (query, key, and value) across all layers. We find that: (1) Object-centric properties are encoded in the similarity maps derived from all three components (q, k, v), unlike prior work that uses only key features or the [CLS] token. (2) This object-centric information is distributed across the network, not just confined to the final layer. Based on these insights, we introduce Object-DINO, a training-free method that extracts this distributed object-centric information. Object-DINO clusters attention heads across all layers based on the similarities of their patches and automatically identifies the object-centric cluster corresponding to all objects. We demonstrate Object-DINO's effectiveness on two applications: enhancing unsupervised object discovery (+3.6 to +12.4 CorLoc gains) and mitigating object hallucination in Multimodal Large Language Models by providing visual grounding. Our results demonstrate that using this distributed object-centric information improves downstream tasks without additional training.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 35 references that are in the map. First posted on arXiv 27 Mar 2026.

Builds on

31 of 35 references.

References · 31

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2023Qwen Technical Reportcited by 3,273
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  10. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  11. 2023A Survey on Multimodal Large Language Modelscited by 898
  12. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
19 more
  1. 2019Are Sixteen Heads Really Better than One?cited by 803
  2. 2020Quantifying Attention Flow in Transformerscited by 759
  3. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  4. 2018Object Hallucination in Image Captioningcited by 661
  5. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  6. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  7. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  8. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  9. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
  10. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 223
  11. 2019Structured Pruning of Large Language Modelscited by 201
  12. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decodingcited by 171
  13. 2022Understanding The Robustness in Vision Transformerscited by 144
  14. 2022TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cutcited by 111
  15. 2020Toward unsupervised, multi-object discovery in large-scale image collectionscited by 65
  16. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidancecited by 44
  17. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  18. 2025Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainabilitycited by 28
  19. 2024RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in Large Vision Language Modelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026What-Where Transformer: A Slot-Centric Visual Backbone for Concurrent Representation and Localizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning Object Focused Attention
  2. 2023Object-centric Learning with Cyclic Walks between Parts and Whole
  3. 2024Bootstrapping Top-down Information for Self-modulating Slot Attention
  4. 2025CObL: Toward Zero-Shot Ordinal Layering without User Prompting
  5. 2024Visual Reasoning in Object-Centric Deep Neural Networks: A Comparative Cognition Approach
  6. 2025LookWhere? Efficient Visual Recognition by Learning Where to Look and What to See from Self-Supervision
  7. 2026Evaluating Object-Centric Models beyond Object Discovery
  8. 2026Learning to Generate Multiple Objects from Dense and Occluded Layouts