mlsanityData to 14 Oct 2026

Upsampling DINOv2 features for unsupervised vision tasks and weakly supervised materials segmentation

arXiv 2410.19836 ↗First posted 20 Oct 2024cs.CV cond-mat.mtrl-sci eess.IVcited by 6

Not on the map. Why?

Abstract (arXiv)

The features of self-supervised vision transformers (ViTs) contain strong semantic and positional information relevant to downstream tasks like object localization and segmentation. Recent works combine these features with traditional methods like clustering, graph partitioning or region correlations to achieve impressive baselines without finetuning or training additional networks. We leverage upsampled features from ViT networks (e.g DINOv2) in two workflows: in a clustering based approach for object localization and segmentation, and paired with standard classifiers in weakly supervised materials segmentation. Both show strong performance on benchmarks, especially in weakly supervised segmentation where the ViT features capture complex relationships inaccessible to classical approaches. We expect the flexibility and generalizability of these features will both speed up and strengthen materials characterization, from segmentation to property-prediction.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 45 of its 77 references that are in the map. First posted on arXiv 20 Oct 2024.

Builds on

45 of 77 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2023Segment Anythingcited by 8,234
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  12. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
33 more
  1. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  4. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  5. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  6. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  7. 2023Vision Transformers Need Registerscited by 871
  8. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  9. 2021Deep ViT Features as Dense Visual Descriptorscited by 330
  10. 2022Masked Autoencoders that Listencited by 273
  11. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  12. 2019CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learningcited by 217
  13. 2021Vision Transformers are Robust Learnerscited by 207
  14. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 167
  15. 2021Self-attention Does Not Need O(n2) Memorycited by 154
  16. 2023A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAScited by 151
  17. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  18. 2022Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformerscited by 118
  19. 2024FeatUp: A Model-Agnostic Framework for Features at Any Resolutioncited by 116
  20. 2022TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cutcited by 113
  21. 2020Object Segmentation Without Labels with Large-Scale Generative Modelscited by 52
  22. 2019A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domainscited by 33
  23. 2022DeepCut: Unsupervised Segmentation using Graph Neural Networks Clusteringcited by 25
  24. 2023MOST: Multiple Object localization with Self-supervised Transformers for object discoverycited by 17
  25. 2023Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Surveycited by 16
  26. 2017Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Networkcited by 11
  27. 2023ScribbleSeg: Scribble-based Interactive Image Segmentationcited by 10
  28. 2023Multi-granularity Interaction Simulation for Unsupervised Interactive Segmentationcited by 9
  29. 2023Understanding Self-Supervised Features for Learning Unsupervised Instance Segmentationcited by 7
  30. 2021Addressing materials' microstructure diversity using transfer learningcited by 4
  31. 2023SAMBA: A Trainable Segmentation Web-App with Smart Labellingcited by 3
  32. 2024SimSAM: Simple Siamese Representations Based Semantic Affinity Matrix for Unsupervised Image Segmentationcited by 2
  33. 2023A Lightweight Clustering Framework for Unsupervised Semantic Segmentationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025Maybe you don't need a U-Net: convolutional feature upsampling for materials micrograph segmentationcited by 2
  2. 2026What DINO saw: ALiBi positional encoding reduces positional bias in Vision Transformerscited by 1
  3. 2026Unlocking Diffusion Hierarchies: Adaptive Timestep Selection for Zero-Shot Segmentationcited by 1
  4. 2026PANC: Prior-Aware Normalized Cut via Anchor-Augmented Token Graphscited by 0
  5. 2026Optimization of Precipitate Segmentation Through Linear Genetic Programming of Image Processingcited by 0
  6. 2026MapSR: Prompt-Driven Land Cover Map Super-Resolution via Vision Foundation Modelscited by 0