mlsanityData to 14 Oct 2026

Understanding Self-Supervised Features for Learning Unsupervised Instance Segmentation

arXiv 2311.14665 ↗First posted 24 Nov 2023cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) can be used to solve complex visual tasks without human labels. Self-supervised representations encode useful semantic information about images, and as a result, they have already been used for tasks such as unsupervised semantic segmentation. In this paper, we investigate self-supervised representations for instance segmentation without any manual annotations. We find that the features of different SSL methods vary in their level of instance-awareness. In particular, DINO features, which are known to be excellent semantic descriptors, lack behind MAE features in their sensitivity for separating instances.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 64 of its 69 references that are in the map. First posted on arXiv 24 Nov 2023.

Builds on

64 of 69 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2017Mask R-CNNcited by 7,277
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  12. 2020Exploring Simple Siamese Representation Learningcited by 2,866
38 more
  1. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  2. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  3. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  4. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  5. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  6. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  7. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  9. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  10. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  11. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  12. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  13. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  14. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  15. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  16. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  17. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  18. 2020Hard Negative Mixing for Contrastive Learningcited by 402
  19. 2021Deep ViT Features as Dense Visual Descriptorscited by 330
  20. 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
  21. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 300
  22. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  23. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  24. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  25. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  26. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  27. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 187
  28. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 167
  29. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  30. 2020BoxInst: High-Performance Instance Segmentation with Box Annotationscited by 151
  31. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  32. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  33. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  34. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  35. 2021Aligning Pretraining for Detection via Object-Level Contrastive Learningcited by 110
  36. 2022FreeSOLO: Learning to Segment Objects without Annotationscited by 102
  37. 2021Instance Localization for Self-supervised Detection Pretrainingcited by 99
  38. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 1.

7 papers.

  1. 2024GLASS: Guided Latent Slot Diffusion for Object-Centric Learningcited by 14
  2. 2025SANSA: Unleashing the Hidden Semantics in SAM2 for Few-Shot Segmentationcited by 8
  3. 2024Upsampling DINOv2 features for unsupervised vision tasks and weakly supervised materials segmentationcited by 6
  4. 2025Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Featurescited by 3
  5. 2024Deep Spectral Improvement for Unsupervised Image Instance Segmentationcited by 1
  6. 2026What DINO saw: ALiBi positional encoding reduces positional bias in Vision Transformerscited by 1
  7. 2025Vision Foundry: A System for Training Foundational Vision AI Modelscited by 0