mlsanityData to 14 Oct 2026

Deep ViT Features as Dense Visual Descriptors

arXiv 2112.05814 ↗First posted 10 Dec 2021cs.CVcited by 330

Not on the map. Why?

Cited by 30 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We study the use of deep features extracted from a pretrained Vision Transformer (ViT) as dense visual descriptors. We observe and empirically demonstrate that such features, when extractedfrom a self-supervised ViT model (DINO-ViT), exhibit several striking properties, including: (i) the features encode powerful, well-localized semantic information, at high spatial granularity, such as object parts; (ii) the encoded semantic information is shared across related, yet different object categories, and (iii) positional bias changes gradually throughout the layers. These properties allow us to design simple methods for a variety of applications, including co-segmentation, part co-segmentation and semantic correspondences. To distill the power of ViT features from convoluted design choices, we restrict ourselves to lightweight zero-shot methodologies (e.g., binning and clustering) applied directly to the features. Since our methods require no additional training nor data, they are readily applicable across a variety of domains. We show by extensive qualitative and quantitative evaluation that our simple methodologies achieve competitive results with recent state-of-the-art supervised methods, and outperform previous unsupervised methods by a large margin. Code is available in dino-vit-features.github.io.

Key facts. Cited by 330 papers in the map (data through 14 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 10 Dec 2021.

Builds on

37 of 62 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  10. 2017Billion-scale similarity search with GPUscited by 2,497
  11. 2017Non-local Neural Networkscited by 2,293
  12. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
25 more
  1. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  2. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  3. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  4. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  5. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 553
  6. 2021Intriguing Properties of Vision Transformerscited by 446
  7. 2019Making Convolutional Networks Shift-Invariant Againcited by 391
  8. 2019On the Relationship between Self-Attention and Convolutional Layerscited by 300
  9. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 232
  10. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  11. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 183
  12. 2019SPair-71k: A Large-scale Benchmark for Semantic Correspondencecited by 146
  13. 2018The Contextual Loss for Image Transformation with Non-Aligned Datacited by 135
  14. 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 114
  15. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  16. 2020Interpretable and Accurate Fine-grained Recognition via Region Groupingcited by 75
  17. 2018Unsupervised Discovery of Object Landmarks as Structural Representationscited by 73
  18. 2017Unsupervised learning of object landmarks by factorized spatial embeddingscited by 64
  19. 2018Deep Feature Factorization For Concept Discoverycited by 64
  20. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  21. 2018Deep Object Co-Segmentationcited by 35
  22. 2020Semantic Pyramid for Image Generationcited by 34
  23. 2021Unsupervised Part Segmentation through Disentangling Appearance and Shapecited by 29
  24. 2021CycleSegNet: Object Co-segmentation with Cycle Refinement and Region Correspondencecited by 19
  25. 2019Deep Object Co-segmentation via Spatial-Semantic Network Modulationcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 27; 2023: 85; 2024: 94; 2025: 70; 2026: 54.

330 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2023LERF: Language Embedded Radiance Fieldscited by 543
  3. 2023Emergent Correspondence from Image Diffusioncited by 446
  4. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  5. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  6. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
  7. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  8. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 301
  9. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 220
  10. 2023EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervisioncited by 217
  11. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 214
  12. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
38 more
  1. 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
  2. 2023Language Embedded 3D Gaussians for Open-Vocabulary Scene Understandingcited by 208
  3. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  4. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 177
  5. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  6. 2023AnyLoc: Towards Universal Visual Place Recognitioncited by 163
  7. 2023SUDS: Scalable Urban Dynamic Scenescited by 151
  8. 2023Weakly Supervised 3D Open-vocabulary Segmentationcited by 135
  9. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  10. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  11. 2024FeatUp: A Model-Agnostic Framework for Features at Any Resolutioncited by 116
  12. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 112
  13. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  14. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  15. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 103
  16. 2023TextDeformer: Geometry Manipulation using Text Guidancecited by 94
  17. 2023Language Embedded Radiance Fields for Zero-Shot Task-Oriented Graspingcited by 91
  18. 2023Telling Left from Right: Identifying Geometry-Aware Semantic Correspondencecited by 86
  19. 2024FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understandingcited by 86
  20. 2024Keypoint Action Tokens Enable In-Context Imitation Learning in Roboticscited by 74
  21. 2022NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenescited by 71
  22. 2024Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulationcited by 71
  23. 2024Improving 2D Feature Representations by 3D-Aware Fine-Tuningcited by 71
  24. 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 66
  25. 2022Unsupervised Object Localization: Observing the Background to Discover Objectscited by 66
  26. 2023LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Groundingcited by 64
  27. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  28. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  29. 2024SpotlessSplats: Ignoring Distractors in 3D Gaussian Splattingcited by 61
  30. 2023Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Modelscited by 59
  31. 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
  32. 2023Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Featurescited by 55
  33. 2024Endora: Video Generation Models as Endoscopy Simulatorscited by 55
  34. 2023Going Denser with Open-Vocabulary Part Segmentationcited by 51
  35. 2024MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereocited by 50
  36. 2023FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Modelscited by 47
  37. 2022Teaching Matters: Investigating the Role of Supervision in Vision Transformerscited by 45
  38. 2023ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformerscited by 45

50 most cited of 330 All 330 (JSON)