mlsanityData to 7 Oct 2026

Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning

arXiv 1908.02983 ↗First posted 8 Aug 2019cs.CVcited by 507

Not on the map. Why?

Abstract (arXiv)

Semi-supervised learning, i.e. jointly learning from labeled and unlabeled samples, is an active research topic due to its key role on relaxing human supervision. In the context of image classification, recent advances to learn from unlabeled samples are mainly focused on consistency regularization methods that encourage invariant predictions for different perturbations of unlabeled samples. We, conversely, propose to learn from unlabeled data by generating soft pseudo-labels using the network predictions. We show that a naive pseudo-labeling overfits to incorrect pseudo-labels due to the so-called confirmation bias and demonstrate that mixup augmentation and setting a minimum number of labeled samples per mini-batch are effective regularization techniques for reducing it. The proposed approach achieves state-of-the-art results in CIFAR-10/100, SVHN, and Mini-ImageNet despite being much simpler than other methods. These results demonstrate that pseudo-labeling alone can outperform consistency regularization methods, while the opposite was supposed in previous work. Source code is available at https://git.io/fjQsC.

Key facts. Cited by 507 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 69; 2022: 100; 2023: 121; 2024: 105; 2025: 66; 2026: 46.

507 papers.

Sort:

By area:

  1. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  2. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  3. 2022Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labelscited by 218
  4. 2021ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentationcited by 184
  5. 2022Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentationcited by 177
  6. 2021Open-World Semi-Supervised Learningcited by 165
  7. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  8. 2023SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learningcited by 133
  9. 2023EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularizationcited by 123
  10. 2021Cycle Self-Training for Domain Adaptationcited by 120
  11. 2021Humble Teachers Teach Better Students for Semi-Supervised Object Detectioncited by 115
  12. 2021Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentationcited by 113
38 more
  1. 2021Instant-Teaching: An End-to-End Semi-Supervised Object Detection Frameworkcited by 112
  2. 2024Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factorscited by 106
  3. 2021Recent advances and clinical applications of deep learning in medical image analysiscited by 104
  4. 2021AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptationcited by 104
  5. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  6. 2022Scribble-Supervised LiDAR Semantic Segmentationcited by 69
  7. 2022Music Source Separation with Band-split RNNcited by 69
  8. 2021Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptationcited by 67
  9. 2022Test-Time Adaptation via Self-Training with Nearest Neighbor Informationcited by 64
  10. 2022Augmentation Matters: A Simple-yet-Effective Approach to Semi-supervised Semantic Segmentationcited by 64
  11. 2022Debiased Self-Training for Semi-Supervised Learningcited by 62
  12. 2022Class-Aware Contrastive Semi-Supervised Learningcited by 62
  13. 2023Cluster-guided Contrastive Graph Clustering Networkcited by 60
  14. 2022Confidence Score for Source-Free Unsupervised Domain Adaptationcited by 58
  15. 2021A Closer Look at Self-training for Zero-Label Semantic Segmentationcited by 57
  16. 2022BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotationcited by 54
  17. 2024AutoEval Done Right: Using Synthetic Data for Model Evaluationcited by 53
  18. 2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
  19. 2021DASO: Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learningcited by 51
  20. 2021Semi-Supervised Action Recognition with Temporal Contrastive Learningcited by 50
  21. 2022Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularizationcited by 50
  22. 2021Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Datacited by 49
  23. 2022Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clusteringcited by 44
  24. 2022Label Matching Semi-Supervised Object Detectioncited by 44
  25. 2022KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learningcited by 41
  26. 2023SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidancecited by 41
  27. 2021LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environmentcited by 40
  28. 2021Self-supervised driven consistency training for annotation efficient histopathology image analysiscited by 39
  29. 2021Self-Tuning for Data-Efficient Deep Learningcited by 39
  30. 2023Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimizationcited by 39
  31. 2021Decoupled Adaptation for Cross-Domain Object Detectioncited by 37
  32. 2022OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learningcited by 37
  33. 2022Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspectivecited by 37
  34. 2024Knowledge Composition using Task Vectors with Learned Anisotropic Scalingcited by 37
  35. 2021ACPL: Anti-curriculum Pseudo-labelling for Semi-supervised Medical Image Classificationcited by 34
  36. 2021Revisiting Consistency Regularization for Semi-Supervised Learningcited by 34
  37. 2023On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansioncited by 34
  38. 2023Towards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalizationcited by 34

50 most cited of 507 All 507 (JSON)