mlsanityData to 7 Oct 2026

In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning

arXiv 2101.06329 ↗First posted 15 Jan 2021cs.LG cs.CVcited by 301

Not on the map. Why?

Abstract (arXiv)

The recent research in semi-supervised learning (SSL) is mostly dominated by consistency regularization based methods which achieve strong performance. However, they heavily rely on domain-specific data augmentations, which are not easy to generate for all data modalities. Pseudo-labeling (PL) is a general SSL approach that does not have this constraint but performs relatively poorly in its original formulation. We argue that PL underperforms due to the erroneous high confidence predictions from poorly calibrated models; these predictions generate many incorrect pseudo-labels, leading to noisy training. We propose an uncertainty-aware pseudo-label selection (UPS) framework which improves pseudo labeling accuracy by drastically reducing the amount of noise encountered in the training process. Furthermore, UPS generalizes the pseudo-labeling process, allowing for the creation of negative pseudo-labels; these negative pseudo-labels can be used for multi-label classification as well as negative learning to improve the single-label classification. We achieve strong performance when compared to recent SSL methods on the CIFAR-10 and CIFAR-100 datasets. Also, we demonstrate the versatility of our method on the video dataset UCF-101 and the multi-label dataset Pascal VOC.

Key facts. Cited by 301 papers in the map (data through 7 Oct 2026). Builds on 49 of its 79 references that are in the map. First posted on arXiv 15 Jan 2021.

Builds on

49 of 79 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  9. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  10. 2016Wide Residual Networkscited by 3,057
  11. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  12. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
37 more
  1. 2016Matching Networks for One Shot Learningcited by 2,246
  2. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  3. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  4. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  5. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  6. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  7. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  8. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  9. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  10. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  11. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  12. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  13. 2015Weight Uncertainty in Neural Networkscited by 554
  14. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  15. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  16. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  17. 2019Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentationcited by 378
  18. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  19. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 304
  20. 2019Label Propagation for Deep Semi-supervised Learningcited by 287
  21. 2018DropBlock: A regularization method for convolutional networkscited by 261
  22. 2019Interpolation Consistency Training for Semi-Supervised Learningcited by 248
  23. 2014Efficient Object Localization Using Convolutional Networkscited by 226
  24. 2020Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentationcited by 222
  25. 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
  26. 2019NLNL: Negative Learning for Noisy Labelscited by 151
  27. 2016Variational Autoencoder for Deep Learning of Images, Labels and Captionscited by 137
  28. 2013Learning by Transductioncited by 120
  29. 2019Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learningcited by 103
  30. 2017Shake-Shake regularizationcited by 87
  31. 2016Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriorscited by 82
  32. 20183D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Trainingcited by 57
  33. 2017Smooth Neighbors on Teacher Graphs for Semi-supervised Learningcited by 48
  34. 2018Deep Metric Transfer for Label Propagation with Limited Annotated Datacited by 33
  35. 2020VideoSSL: Semi-Supervised Learning for Video Classificationcited by 27
  36. 2019Distance-Based Learning from Errors for Confidence Calibrationcited by 23
  37. 2019Repetitive Reprediction Deep Decipher for Semi-Supervised Learningcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 83; 2023: 81; 2024: 57; 2025: 37; 2026: 14.

301 papers.

Sort:

By area:

  1. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  2. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  3. 2022Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labelscited by 218
  4. 2024Efficient Test-Time Adaptation of Vision-Language Modelscited by 138
  5. 2022Unsupervised Prompt Learning for Vision-Language Modelscited by 125
  6. 2021Mutual Consistency Learning for Semi-supervised Medical Image Segmentationcited by 107
  7. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 102
  8. 2021Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibrationcited by 91
  9. 2022UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learningcited by 77
  10. 2022Debiased Self-Training for Semi-Supervised Learningcited by 62
  11. 2022A Close Look into the Calibration of Pre-trained Language Modelscited by 57
  12. 2021Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advancescited by 54
38 more
  1. 2023Better Zero-Shot Reasoning with Self-Adaptive Promptingcited by 52
  2. 2022Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detectioncited by 47
  3. 2022Semi-supervised 3D Object Detection with Proficient Teacherscited by 45
  4. 2021Structured Sparse R-CNN for Direct Scene Graph Generationcited by 44
  5. 2022KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learningcited by 41
  6. 2021Learning Fair Classifiers with Partially Annotated Group Labelscited by 40
  7. 2022Parametric Information Maximization for Generalized Category Discoverycited by 39
  8. 2022OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learningcited by 37
  9. 2023Continual Test-time Domain Adaptation via Dynamic Sample Selectioncited by 35
  10. 2021ACPL: Anti-curriculum Pseudo-labelling for Semi-supervised Medical Image Classificationcited by 34
  11. 2021Reference-guided Pseudo-Label Generation for Medical Semantic Segmentationcited by 33
  12. 2022Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervisioncited by 33
  13. 2022Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundariescited by 31
  14. 2022BoxTeacher: Exploring High-Quality Pseudo Labels for Weakly Supervised Instance Segmentationcited by 31
  15. 2022Uncertainty-Aware Learning Against Label Noise on Imbalanced Datasetscited by 28
  16. 2025Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Priorcited by 28
  17. 2022Towards Realistic Semi-Supervised Learningcited by 27
  18. 2021Learning from Temporal Gradient for Semi-supervised Action Recognitioncited by 26
  19. 2023Boosting Semi-Supervised Learning by Exploiting All Unlabeled Datacited by 26
  20. 2023CiPR: An Efficient Framework with Cross-instance Positive Relations for Generalized Category Discoverycited by 26
  21. 2022NP-Match: When Neural Processes meet Semi-Supervised Learningcited by 25
  22. 2022ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularizationcited by 25
  23. 2023Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modelingcited by 25
  24. 2023Incremental Generalized Category Discoverycited by 25
  25. 2022Acknowledging the Unknown for Multi-label Learning with Single Positive Labelscited by 24
  26. 2022CoNMix for Source-free Single and Multi-target Domain Adaptationcited by 23
  27. 2022MutexMatch: Semi-Supervised Learning with Mutex-Based Consistency Regularizationcited by 22
  28. 2022FedMix: Mixed Supervised Federated Learning for Medical Image Segmentationcited by 22
  29. 2023Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learningcited by 22
  30. 2021Not All Relations are Equal: Mining Informative Labels for Scene Graph Generationcited by 21
  31. 2022Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin Pickingcited by 20
  32. 2023FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learningcited by 20
  33. 2024Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labelingcited by 20
  34. 2021Self-training Converts Weak Learners to Strong Learners in Mixture Modelscited by 19
  35. 2022Fuzzy Positive Learning for Semi-supervised Semantic Segmentationcited by 19
  36. 2024Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learningcited by 19
  37. 2021Combating Noise: Semi-supervised Learning by Region Uncertainty Quantificationcited by 18
  38. 2022An Information-theoretical Approach to Semi-supervised Learning under Covariate-shiftcited by 18

50 most cited of 301 All 301 (JSON)