Active Semi-Supervised Learning by Exploring Per-Sample Uncertainty and Consistency
Not on the map. Why?
Abstract (arXiv)
Active Learning (AL) and Semi-supervised Learning are two techniques that have been studied to reduce the high cost of deep learning by using a small amount of labeled data and a large amount of unlabeled data. To improve the accuracy of models at a lower cost, we propose a method called Active Semi-supervised Learning (ASSL), which combines AL and SSL. To maximize the synergy between AL and SSL, we focused on the differences between ASSL and AL. ASSL involves more dynamic model updates than AL due to the use of unlabeled data in the training process, resulting in the temporal instability of the predicted probabilities of the unlabeled data. This makes it difficult to determine the true uncertainty of the unlabeled data in ASSL. To address this, we adopted techniques such as exponential moving average (EMA) and upper confidence bound (UCB) used in reinforcement learning. Additionally, we analyzed the effect of label noise on unsupervised learning by using weak and strong augmentation pairs to address datainconsistency. By considering both uncertainty and datainconsistency, we acquired data samples that were used in the proposed ASSL method. Our experiments showed that ASSL achieved about 5.3 times higher computational efficiency than SSL while achieving the same performance, and it outperformed the state-of-the-art AL method.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 15 Mar 2023.
Builds on
34 of 48 references.
References · 34
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2020Supervised Contrastive Learningcited by 2,803
- 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
- 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
- 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
- 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
- 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
- 2017Deep Bayesian Active Learning with Image Datacited by 812
- 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
- 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
- 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
22 more
- 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
- 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
- 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
- 2019Learning Loss for Active Learningcited by 335
- 2019Variational Adversarial Active Learningcited by 284
- 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
- 2020Contextual Diversity for Active Learningcited by 139
- 2019Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Costcited by 114
- 2021Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentationcited by 113
- 2020Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learningcited by 105
- 2019Deep Active Learning: Unified and Principled Method for Query and Trainingcited by 92
- 2020Task-Aware Variational Adversarial Active Learningcited by 84
- 2021Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learningcited by 78
- 2022Active Learning by Feature Mixingcited by 63
- 2022Semi-supervised Vision Transformers at Scalecited by 49
- 2021Accelerating Deep Learning with Dynamic Data Pruningcited by 43
- 2021Not All Labels Are Equal: Rationalizing The Labeling Costs for Training Object Detectioncited by 27
- 2022Contrastive Regularization for Semi-Supervised Learningcited by 24
- 2020An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human Pose Estimationcited by 21
- 2019Combining MixMatch and Active Learning for Better Accuracy with Fewer Labelscited by 17
- 2021A Neural Pre-Conditioning Active Learning Algorithm to Reduce Label Complexitycited by 7
- 2022Pose-MUM : Reinforcing Key Points Relationship for Semi-Supervised Human Pose Estimationcited by 3
Built on by
Citing papers per year: 2025: 2.
2 papers.