mlsanityData to 14 Oct 2026

Semi-supervised learning made simple with self-supervised clustering

arXiv 2306.07483 ↗First posted 13 Jun 2023cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning models have been shown to learn rich visual representations without requiring human annotations. However, in many real-world scenarios, labels are partially available, motivating a recent line of work on semi-supervised methods inspired by self-supervised principles. In this paper, we propose a conceptually simple yet empirically powerful approach to turn clustering-based self-supervised methods such as SwAV or DINO into semi-supervised learners. More precisely, we introduce a multi-task framework merging a supervised objective using ground-truth labels and a self-supervised objective relying on clustering assignments with a single cross-entropy loss. This approach may be interpreted as imposing the cluster centroids to be class prototypes. Despite its simplicity, we provide empirical evidence that our approach is highly effective and achieves state-of-the-art performance on CIFAR100 and ImageNet.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 64 of its 80 references that are in the map. First posted on arXiv 13 Jun 2023.

Builds on

64 of 80 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  10. 2014Going Deeper with Convolutionscited by 4,241
  11. 2016Wide Residual Networkscited by 3,070
  12. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
38 more
  1. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  3. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  4. 2020Supervised Contrastive Learningcited by 2,829
  5. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  6. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  7. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  10. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  11. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  13. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  14. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  15. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  16. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  17. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  18. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  19. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 540
  20. 2017Large Batch Training of Convolutional Networkscited by 535
  21. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  22. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  23. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  24. 2020Meta Pseudo Labelscited by 374
  25. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  26. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  27. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 326
  28. 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
  29. 2020Whitening for Self-Supervised Representation Learningcited by 279
  30. 2019Interpolation Consistency Training for Semi-Supervised Learningcited by 251
  31. 2020CoMatch: Semi-supervised Learning with Contrastive Graph Regularizationcited by 167
  32. 2021A Unified Objective for Novel Class Discoverycited by 162
  33. 2021Self-Supervised Models are Continual Learnerscited by 149
  34. 2021Dash: Semi-Supervised Learning with Dynamic Thresholdingcited by 148
  35. 2018Deep Co-Training for Semi-Supervised Image Recognitioncited by 147
  36. 2021Neighborhood Contrastive Learning for Novel Class Discoverycited by 121
  37. 2020Boosting Contrastive Self-Supervised Learning with False Negative Cancellationcited by 119
  38. 2022SimMatch: Semi-supervised Learning with Similarity Matchingcited by 118

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 10; 2025: 6; 2026: 2.

20 papers.

Sort:

By area:

  1. 2024mathbbX-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphscited by 9
  2. 2024A Survey of the Self Supervised Learning Mechanisms for Vision Transformerscited by 9
  3. 2023Toulouse Hyperspectral Data Set: a benchmark data set to assess semi-supervised spectral representation learning and pixel-wise classification techniquescited by 8
  4. 2025Self-Reinforcing Prototype Evolution with Dual-Knowledge Cooperation for Semi-Supervised Lifelong Person Re-Identificationcited by 7
  5. 2025MM-DINOv2: Adapting Foundation Models for Multi-Modal Medical Image Analysiscited by 7
  6. 2024Center-Based Relaxed Learning Against Membership Inference Attackscited by 4
  7. 2025TRiCo: Triadic Game-Theoretic Co-Training for Robust Semi-Supervised Learningcited by 4
  8. 2023RankMatch: A Novel Approach to Semi-Supervised Label Distribution Learning Leveraging Inter-label Correlationscited by 3
  9. 2025K*-Means: A Parameter-free Clustering Algorithmcited by 3
  10. 2025A Hidden Stumbling Block in Generalized Category Discovery: Distracted Attentioncited by 3
  11. 2024Is user feedback always informative? Retrieval Latent Defending for Semi-Supervised Domain Adaptation without Source Datacited by 2
  12. 2024A Unified Contrastive Loss for Self-Trainingcited by 2
8 more
  1. 2026Towards Multimodal Domain Generalization with Few Labelscited by 2
  2. 2024Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorizationcited by 1
  3. 2024A Method of Moments Embedding Constraint and its Application to Semi-Supervised Learningcited by 1
  4. 2024AnomalyAID: Reliable Interpretation for Semi-supervised Network Anomaly Detectioncited by 1
  5. 2024Rethinking Open-World Semi-Supervised Learning: Distribution Mismatch and Inductive Inferencecited by 0
  6. 2024Learning Unlabeled Clients Divergence for Federated Semi-Supervised Learning via Anchor Model Aggregationcited by 0
  7. 2025Learn Faster and Remember More: Balancing Exploration and Exploitation for Continual Test-time Adaptationcited by 0
  8. 2026JEPAMatch: Geometric Representation Shaping for Semi-Supervised Learningcited by 0