mlsanityData to 7 Oct 2026

Unsupervised Selective Labeling for More Effective Semi-Supervised Learning

arXiv 2110.03006 ↗First posted 6 Oct 2021cs.LG cs.CVcited by 27

Not on the map. Why?

Abstract (arXiv)

Given an unlabeled dataset and an annotation budget, we study how to selectively label a fixed number of instances so that semi-supervised learning (SSL) on such a partially labeled dataset is most effective. We focus on selecting the right data to label, in addition to usual SSL's propagating labels from labeled data to the rest unlabeled data. This instance selection task is challenging, as without any labeled data we do not know what the objective of learning should be. Intuitively, no matter what the downstream task is, instances to be labeled must be representative and diverse: The former would facilitate label propagation to unlabeled data, whereas the latter would ensure coverage of the entire dataset. We capture this idea by selecting cluster prototypes, either in a pretrained feature space, or along with feature optimization, both without labels. Our unsupervised selective labeling consistently improves SSL methods over state-of-the-art active learning given labeled data, by 8 to 25 times in label efficiency. For example, it boosts FixMatch by 10% (14%) in accuracy on CIFAR-10 (ImageNet-1K) with 0.08% (0.2%) labeled data, demonstrating that small computation spent on selecting what data to label brings significant gain especially under a low annotation budget. Our work sets a new standard for practical and efficient SSL.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 50 of its 87 references that are in the map. First posted on arXiv 6 Oct 2021.

Builds on

50 of 87 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  12. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
38 more
  1. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  2. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  3. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  4. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  5. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  6. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  7. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  8. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  9. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  10. 2020Long-tail learning via logit adjustmentcited by 605
  11. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
  12. 2020A Survey of Deep Active Learningcited by 547
  13. 2017Large Batch Training of Convolutional Networkscited by 529
  14. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 470
  15. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  16. 2020SCAN: Learning to Classify Images without Labelscited by 351
  17. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  18. 2019Learning Loss for Active Learningcited by 335
  19. 2019Variational Adversarial Active Learningcited by 284
  20. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  21. 2020CoMatch: Semi-supervised Learning with Contrastive Graph Regularizationcited by 167
  22. 2018Adversarial Active Learning for Deep Networks: a Margin Based Approachcited by 155
  23. 2019Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Costcited by 114
  24. 2021Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samplescited by 114
  25. 2021Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learningcited by 78
  26. 2020Sequential Graph Convolutional Network for Active Learningcited by 69
  27. 2020Unsupervised Feature Learning by Cross-Level Instance-Group Discriminationcited by 51
  28. 2020Improving Unsupervised Image Clustering With Robust Learningcited by 51
  29. 2018Deep k-Means: Jointly clustering with k-Means and learning representationscited by 50
  30. 2021Semi-Supervised Active Learning with Temporal Output Discrepancycited by 50
  31. 2019Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applicationscited by 43
  32. 2018Deep Metric Transfer for Label Propagation with Limited Annotated Datacited by 33
  33. 2021Iterative Human and Automated Identification of Wildlife Imagescited by 22
  34. 2020On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake?cited by 18
  35. 2021MCDAL: Maximum Classifier Discrepancy for Active Learningcited by 18
  36. 2019Combining MixMatch and Active Learning for Better Accuracy with Fewer Labelscited by 17
  37. 2021Improve Unsupervised Pretraining for Few-label Transfercited by 15
  38. 2020Unsupervised Image Classification for Deep Representation Learningcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 13; 2024: 6; 2025: 5; 2026: 2.

27 papers.

Sort:

By area:

  1. 2023Data Selection for Language Models via Importance Resamplingcited by 322
  2. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 266
  3. 2023Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labelscited by 37
  4. 2023Unsupervised Universal Image Segmentationcited by 34
  5. 2023A comprehensive survey on deep active learning in medical image analysiscited by 32
  6. 2023Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learningcited by 17
  7. 2023InstanT: Semi-supervised Learning with Instance-dependent Thresholdscited by 13
  8. 2024Visual Prompt Selection for In-Context Learning Segmentationcited by 12
  9. 2023Towards Free Data Selection with General-Purpose Modelscited by 10
  10. 2025DataS^3: Dataset Subset Selection for Specializationcited by 6
  11. 2023Data-Centric Digital Agriculture: A Perspectivecited by 5
  12. 2024Annotations on a Budget: Leveraging Geo-Data Similarity to Balance Model Performance and Annotation Costcited by 5
15 more
  1. 2022Active Self-Semi-Supervised Learning for Few Labeled Samplescited by 4
  2. 2023A Survey of Historical Learning: Learning Models with Learning Historycited by 3
  3. 2023UP-DP: Unsupervised Prompt Learning for Data Pre-Selection with Vision-Language Modelscited by 3
  4. 2024OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learningcited by 3
  5. 2023A Mixture of Exemplars Approach for Efficient Out-of-Distribution Detection with Foundation Modelscited by 2
  6. 2024Enhancing Semi-Supervised Learning via Representative and Diverse Sample Selectioncited by 2
  7. 2025Few-Shot Learning in Video and 3D Object Detection: A Surveycited by 2
  8. 2025Model-agnostic Selective Labeling with Provable Statistical Guaranteescited by 2
  9. 2023Cold PAWS: Unsupervised class discovery and addressing the cold-start problem for semi-supervised learningcited by 1
  10. 2025Sharpness-aware Dynamic Anchor Selection for Generalized Category Discoverycited by 1
  11. 2026LUCoS: Latent Unsupervised Context Selection for Tabular Foundation Modelscited by 1
  12. 2024Diverse Teacher-Students for Deep Safe Semi-Supervised Learning under Class Mismatchcited by 0
  13. 2024Towards Clinical Practice in CT-Based Pulmonary Disease Screening: An Efficient and Reliable Frameworkcited by 0
  14. 2025Adapting Medical Vision Foundation Models for Volumetric Medical Image Segmentation via Active Learning and Selective Semi-supervised Fine-tuningcited by 0
  15. 2026Dataset-Aware Cold-Start Active Learning for Annotation-Efficient 3D Medical Image Segmentationcited by 0