# Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

arXiv: https://arxiv.org/abs/2303.11066 · HTML: https://mlsanity.com/paper/2303.11066/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 49 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Semi-Supervised Learning with Ladder Networks](https://mlsanity.com/paper/1507.02672/) (2015)
- [Learning with Pseudo-Ensembles](https://mlsanity.com/paper/1412.4864/) (2014)
- [FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning](https://mlsanity.com/paper/2205.07246/) (2022)
- [Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels](https://mlsanity.com/paper/2203.03884/) (2022)
- [ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2106.05095/) (2021)
- [CoMatch: Semi-supervised Learning with Contrastive Graph Regularization](https://mlsanity.com/paper/2011.11183/) (2020)
- [Dash: Semi-Supervised Learning with Dynamic Thresholding](https://mlsanity.com/paper/2109.00650/) (2021)
- [SimMatch: Semi-supervised Learning with Similarity Matching](https://mlsanity.com/paper/2203.06915/) (2022)
- [DMT: Dynamic Mutual Training for Semi-Supervised Learning](https://mlsanity.com/paper/2004.08514/) (2020)
- [BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotation](https://mlsanity.com/paper/2203.02533/) (2022)
- [Dense Teacher: Dense Pseudo-Labels for Semi-supervised Object Detection](https://mlsanity.com/paper/2207.02541/) (2022)
- [SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06480/) (2021)
- [All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training](https://mlsanity.com/paper/2104.05248/) (2021)
- [NP-Match: When Neural Processes meet Semi-Supervised Learning](https://mlsanity.com/paper/2207.01066/) (2022)
- [Contrastive Regularization for Semi-Supervised Learning](https://mlsanity.com/paper/2201.06247/) (2022)
- [Mutual Exclusivity Loss for Semi-Supervised Deep Learning](https://mlsanity.com/paper/1606.03141/) (2016)
- [AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence](https://mlsanity.com/paper/2011.11779/) (2020)

## Built on by (most cited first, 26 of 26)

- [CGMatch: A Different Perspective of Semi-supervised Learning](https://mlsanity.com/paper/2503.02231/) (2025)
- [MOS: Modeling Object-Scene Associations in Generalized Category Discovery](https://mlsanity.com/paper/2503.12035/) (2025)
- [Foster Adaptivity and Balance in Learning with Noisy Labels](https://mlsanity.com/paper/2407.02778/) (2024)
- [Self-Reinforcing Prototype Evolution with Dual-Knowledge Cooperation for Semi-Supervised Lifelong Person Re-Identification](https://mlsanity.com/paper/2507.01884/) (2025)
- [OTMatch: Improving Semi-Supervised Learning with Optimal Transport](https://mlsanity.com/paper/2310.17455/) (2023)
- [AllMatch: Exploiting All Unlabeled Data for Semi-Supervised Learning](https://mlsanity.com/paper/2406.15763/) (2024)
- [Divide-and-Conquer for Enhancing Unlabeled Learning, Stability, and Plasticity in Semi-supervised Continual Learning](https://mlsanity.com/paper/2508.05316/) (2025)
- [Semi-Supervised Unconstrained Head Pose Estimation in the Wild](https://mlsanity.com/paper/2404.02544/) (2024)
- [Deep Mutual Learning among Partially Labeled Datasets for Multi-Organ Segmentation](https://mlsanity.com/paper/2407.12611/) (2024)
- [Is user feedback always informative? Retrieval Latent Defending for Semi-Supervised Domain Adaptation without Source Data](https://mlsanity.com/paper/2407.15383/) (2024)
- [Improving 3D Semi-supervised Learning by Effectively Utilizing All Unlabelled Data](https://mlsanity.com/paper/2409.13977/) (2024)
- [FATE: A Prompt-Tuning-Based Semi-Supervised Learning Framework for Extremely Limited Labeled Data](https://mlsanity.com/paper/2504.09828/) (2025)
- [Boosting Semi-Supervised Temporal Action Localization by Learning from Non-Target Classes](https://mlsanity.com/paper/2403.11189/) (2024)
- [TACLE: Task and Class-aware Exemplar-free Semi-supervised Class Incremental Learning](https://mlsanity.com/paper/2407.08041/) (2024)
- [Iterative pseudo-labeling based adaptive copy-paste supervision for semi-supervised tumor segmentation](https://mlsanity.com/paper/2508.04044/) (2025)
- [Rethinking Scale Imbalance in Semi-supervised Object Detection for Aerial Images](https://mlsanity.com/paper/2310.14718/) (2023)
- [Feature Space Renormalization for Semi-supervised Learning](https://mlsanity.com/paper/2311.04055/) (2023)
- [Analyzing Decades-Long Environmental Changes in Namibia Using Archival Aerial Photography and Deep Learning](https://mlsanity.com/paper/2404.08544/) (2024)
- [Smooth Pseudo-Labeling](https://mlsanity.com/paper/2405.14313/) (2024)
- [Semi-Supervised Audio-Visual Video Action Recognition with Audio Source Localization Guided Mixup](https://mlsanity.com/paper/2503.02284/) (2025)
- [Domain-Agnostic Feature Modulation for Semi-Supervised Domain Generalization](https://mlsanity.com/paper/2503.20897/) (2025)
- [CaliMatch: Adaptive Calibration for Improving Safe Semi-supervised Learning](https://mlsanity.com/paper/2508.00922/) (2025)
- [CoVar: Confidence-Variance-Guided Pseudo-Label Selection for Semi-Supervised Learning](https://mlsanity.com/paper/2601.11670/) (2026)
- [SemAlign: Language Guided Semi-supervised Domain Generalization](https://mlsanity.com/paper/2601.11724/) (2026)
- [PRISM: Prioritized Channel Importance with Semi-supervised Domain Adaptation for Cross-Subject EEG Emotion Recognition](https://mlsanity.com/paper/2607.00358/) (2026)
- [MTSSL: Meta-Thresholding Semi-Supervised Learning](https://mlsanity.com/paper/2607.16363/) (2026)

## Abstract (arXiv)

Semi-supervised learning (SSL) has attracted enormous attention due to its vast potential of mitigating the dependence on large labeled datasets. The latest methods (e.g., FixMatch) use a combination of consistency regularization and pseudo-labeling to achieve remarkable successes. However, these methods all suffer from the waste of complicated examples since all pseudo-labels have to be selected by a high threshold to filter out noisy ones. Hence, the examples with ambiguous predictions will not contribute to the training phase. For better leveraging all unlabeled examples, we propose two novel techniques: Entropy Meaning Loss (EML) and Adaptive Negative Learning (ANL). EML incorporates the prediction distribution of non-target classes into the optimization objective to avoid competition with target class, and thus generating more high-confidence predictions for selecting pseudo-label. ANL introduces the additional negative pseudo-label for all unlabeled data to leverage low-confidence examples. It adaptively allocates this label by dynamically evaluating the top-k performance of the model. EML and ANL do not introduce any additional parameter and hyperparameter. We integrate these techniques with FixMatch, and develop a simple yet powerful framework called FullMatch. Extensive experiments on several common SSL benchmarks (CIFAR-10/100, SVHN, STL-10 and ImageNet) demonstrate that FullMatch exceeds FixMatch by a large margin. Integrated with FlexMatch (an advanced FixMatch-based framework), we achieve state-of-the-art performance. Source code is at https://github.com/megvii-research/FullMatch.