# Boosting Semi-Supervised Learning by bridging high and low-confidence predictions

arXiv: https://arxiv.org/abs/2308.07509 · HTML: https://mlsanity.com/paper/2308.07509/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 56 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning](https://mlsanity.com/paper/1606.04586/) (2016)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [Learning with Pseudo-Ensembles](https://mlsanity.com/paper/1412.4864/) (2014)
- [CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2102.09559/) (2021)
- [Exploring the Limits of Language Modeling](https://mlsanity.com/paper/1602.02410/) (2016)
- [CoMatch: Semi-supervised Learning with Contrastive Graph Regularization](https://mlsanity.com/paper/2011.11183/) (2020)
- [FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2011.09230/) (2020)
- [SimMatch: Semi-supervised Learning with Similarity Matching](https://mlsanity.com/paper/2203.06915/) (2022)
- [Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning](https://mlsanity.com/paper/2007.08844/) (2020)
- [AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation](https://mlsanity.com/paper/2106.04732/) (2021)
- [Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning](https://mlsanity.com/paper/2101.08482/) (2021)
- [DMT: Dynamic Mutual Training for Semi-Supervised Learning](https://mlsanity.com/paper/2004.08514/) (2020)

## Built on by (most cited first, 11 of 11)

- [SequenceMatch: Revisiting the design of weak-strong augmentations for Semi-supervised learning](https://mlsanity.com/paper/2310.15787/) (2023)
- [Revisiting semi-supervised learning in the era of foundation models](https://mlsanity.com/paper/2503.09707/) (2025)
- [(FL)2: Overcoming Few Labels in Federated Semi-Supervised Learning](https://mlsanity.com/paper/2410.23227/) (2024)
- [CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections](https://mlsanity.com/paper/2411.19346/) (2024)
- [FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients](https://mlsanity.com/paper/2402.10191/) (2024)
- [Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptation](https://mlsanity.com/paper/2608.09052/) (2026)
- [LayerMatch: Do Pseudo-labels Benefit All Layers?](https://mlsanity.com/paper/2406.14207/) (2024)
- [CoVLM: Leveraging Consensus from Vision-Language Models for Semi-supervised Multi-modal Fake News Detection](https://mlsanity.com/paper/2410.04426/) (2024)
- [Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Model](https://mlsanity.com/paper/2509.03895/) (2025)
- [SpikeMatch: Semi-Supervised Learning with Temporal Dynamics of Spiking Neural Networks](https://mlsanity.com/paper/2509.22581/) (2025)
- [MTSSL: Meta-Thresholding Semi-Supervised Learning](https://mlsanity.com/paper/2607.16363/) (2026)

## Abstract (arXiv)

Pseudo-labeling is a crucial technique in semi-supervised learning (SSL), where artificial labels are generated for unlabeled data by a trained model, allowing for the simultaneous training of labeled and unlabeled data in a supervised setting. However, several studies have identified three main issues with pseudo-labeling-based approaches. Firstly, these methods heavily rely on predictions from the trained model, which may not always be accurate, leading to a confirmation bias problem. Secondly, the trained model may be overfitted to easy-to-learn examples, ignoring hard-to-learn ones, resulting in the textit"Matthew effect" where the already strong become stronger and the weak weaker. Thirdly, most of the low-confidence predictions of unlabeled data are discarded due to the use of a high threshold, leading to an underutilization of unlabeled data during training. To address these issues, we propose a new method called ReFixMatch, which aims to utilize all of the unlabeled data during training, thus improving the generalizability of the model and performance on SSL benchmarks. Notably, ReFixMatch achieves 41.05% top-1 accuracy with 100k labeled examples on ImageNet, outperforming the baseline FixMatch and current state-of-the-art methods.