mlsanityData to 7 Oct 2026

Self-labelling via simultaneous clustering and representation learning

arXiv 1911.05371 ↗First posted 13 Nov 2019cs.CV cs.NEcited by 525

Not on the map. Why?

Abstract (arXiv)

Combining clustering and representation learning is one of the most promising approaches for unsupervised learning of deep neural networks. However, doing so naively leads to ill posed learning problems with degenerate solutions. In this paper, we propose a novel and principled learning formulation that addresses these issues. The method is obtained by maximizing the information between labels and input data indices. We show that this criterion extends standard crossentropy minimization to an optimal transport problem, which we solve efficiently for millions of input images and thousands of labels using a fast variant of the Sinkhorn-Knopp algorithm. The resulting method is able to self-label visual data so as to train highly competitive image representations without manual labels. Our method achieves state of the art representation learning performance for AlexNet and ResNet-50 on SVHN, CIFAR-10, CIFAR-100 and ImageNet and yields the first self-supervised AlexNet that outperforms the supervised Pascal VOC detection baseline. Code and models are available.

Key facts. Cited by 525 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 129; 2022: 127; 2023: 112; 2024: 91; 2025: 50; 2026: 16.

525 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  4. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  5. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  6. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  7. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  8. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  9. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  10. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  11. 2021Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentationcited by 268
  12. 2022Contrastive Test-Time Adaptationcited by 258
38 more
  1. 2023A Cookbook of Self-Supervised Learningcited by 248
  2. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
  3. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  4. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 197
  5. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  6. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 186
  7. 2021Reinforcement Learning with Prototypical Representationscited by 179
  8. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  9. 2021A Unified Objective for Novel Class Discoverycited by 161
  10. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  11. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  12. 2022Rethinking Semantic Segmentation: A Prototype Viewcited by 140
  13. 2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
  14. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  15. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  16. 2022SimMatch: Semi-supervised Learning with Similarity Matchingcited by 118
  17. 2021Are Large-scale Datasets Necessary for Self-Supervised Pre-training?cited by 117
  18. 2022What to Hide from Your Students: Attention-Guided Masked Image Modelingcited by 117
  19. 2021Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samplescited by 114
  20. 2022Parametric Classification for Generalized Category Discovery: A Baseline Studycited by 113
  21. 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
  22. 2021Efficient Training of Visual Transformers with Small Datasetscited by 107
  23. 2021SPICE: Semantic Pseudo-labeling for Image Clusteringcited by 104
  24. 2021Pre-training without Natural Imagescited by 100
  25. 2021Contrastive Learning with Stronger Augmentationscited by 100
  26. 2021SiT: Self-supervised vIsion Transformercited by 99
  27. 2022OmniMAE: Single Model Masked Pretraining on Images and Videoscited by 99
  28. 2021Large-scale Unsupervised Semantic Segmentationcited by 98
  29. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  30. 2022GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Modelscited by 87
  31. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
  32. 2022Temporal Alignment Networks for Long-term Videocited by 84
  33. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  34. 2021Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptationcited by 83
  35. 2021ReSSL: Relational Self-Supervised Learning with Weak Augmentationcited by 83
  36. 2021Weakly Supervised Contrastive Learningcited by 81
  37. 2021Robust Audio-Visual Instance Discriminationcited by 80
  38. 2022Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervisioncited by 80

50 most cited of 525 All 525 (JSON)