How To Overcome Confirmation Bias in Semi-Supervised Image Classification By Active Learning
Not on the map. Why?
Abstract (arXiv)
Do we need active learning? The rise of strong deep semi-supervised methods raises doubt about the usability of active learning in limited labeled data settings. This is caused by results showing that combining semi-supervised learning (SSL) methods with a random selection for labeling can outperform existing active learning (AL) techniques. However, these results are obtained from experiments on well-established benchmark datasets that can overestimate the external validity. However, the literature lacks sufficient research on the performance of active semi-supervised learning methods in realistic data scenarios, leaving a notable gap in our understanding. Therefore we present three data challenges common in real-world applications: between-class imbalance, within-class imbalance, and between-class similarity. These challenges can hurt SSL performance due to confirmation bias. We conduct experiments with SSL and AL on simulated data challenges and find that random sampling does not mitigate confirmation bias and, in some cases, leads to worse performance than supervised learning. In contrast, we demonstrate that AL can overcome confirmation bias in SSL in these realistic settings. Our results provide insights into the potential of combining active and semi-supervised learning in the presence of common real-world challenges, which is a promising direction for robust methods when learning with limited labeled data in real-world applications.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 45 of its 77 references that are in the map. First posted on arXiv 16 Aug 2023.
Builds on
45 of 77 references.
References · 45
- 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
- 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
- 2017Deep Bayesian Active Learning with Image Datacited by 819
- 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
- 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 610
- 2020A Survey of Deep Active Learningcited by 554
- 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 540
- 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
- 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
- 2019Variational Adversarial Active Learningcited by 284
- 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
- 2022The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluationcited by 139
33 more
- 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 133
- 2019Image Classification with Deep Learning in the Presence of Noisy Labels: A Surveycited by 101
- 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
- 2019Diverse mini-batch Active Learningcited by 89
- 2022A Comparative Survey of Deep Active Learningcited by 89
- 2021SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarioscited by 84
- 2020Active Domain Adaptation via Clustering Uncertainty-weighted Embeddingscited by 72
- 2018Practical Obstacles to Deploying Active Learningcited by 68
- 2020State-Relabeling Adversarial Active Learningcited by 60
- 2021Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentationcited by 57
- 2020Towards Robust and Reproducible Active Learning Using Neural Networkscited by 54
- 2022BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotationcited by 54
- 2019Semantic Redundancies in Image-Classification Datasets: The 10% You Don't Needcited by 43
- 2021Reducing Label Effort: Self-Supervised meets Active Learningcited by 42
- 2019Parting with Illusions about Deep Active Learningcited by 38
- 2021Effective Evaluation of Deep Active Learning on Image Classification Taskscited by 37
- 2020ALICE: Active Learning with Contrastive Natural Language Explanationscited by 36
- 2022Active Learning for Open-set Annotationcited by 34
- 2020Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervisioncited by 32
- 2022GALAXY: Graph-based Active Learning at the Extremecited by 29
- 2020Class-Imbalanced Semi-Supervised Learningcited by 28
- 2022PT4AL: Using Self-Supervised Pretext Tasks for Active Learningcited by 27
- 2022In Defense of Core-set: A Density-aware Core-set Selection for Active Learningcited by 27
- 2020VaB-AL: Incorporating Class Imbalance and Difficulty with Variational Bayes for Active Learningcited by 25
- 2022Meta-Query-Net: Resolving Purity-Informativeness Dilemma in Open-set Active Learningcited by 25
- 2022A Lagrangian Duality Approach to Active Learningcited by 24
- 2021Boosting Active Learning via Improving Test Performancecited by 19
- 2020On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake?cited by 18
- 2021Class-Balanced Active Learning for Image Classificationcited by 15
- 2019Learning to Sample: an Active Learning Frameworkcited by 11
- 2021Improving Model Robustness by Adaptively Correcting Perturbation Levels with Active Queriescited by 9
- 2018Learning from Large-scale Noisy Web Data with Ubiquitous Reweighting for Image Classificationcited by 7
- 2021Tackling Inter-Class Similarity and Intra-Class Variance for Microscopic Image-based Classificationcited by 6
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.