# Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning

arXiv: https://arxiv.org/abs/2001.06001 · HTML: https://mlsanity.com/paper/2001.06001/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2020.

## Built on by (most cited first, 14 of 14)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias](https://mlsanity.com/paper/2403.07857/) (2024)
- [A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification](https://mlsanity.com/paper/2104.00679/) (2021)
- [Reference-guided Pseudo-Label Generation for Medical Semantic Segmentation](https://mlsanity.com/paper/2112.00735/) (2021)
- [The Semi-Supervised iNaturalist-Aves Challenge at FGVC7 Workshop](https://mlsanity.com/paper/2103.06937/) (2021)
- [A Review of Pseudo-Labeling for Computer Vision](https://mlsanity.com/paper/2408.07221/) (2024)
- [Semi-Supervised Learning with Taxonomic Labels](https://mlsanity.com/paper/2111.11595/) (2021)
- [Losses over Labels: Weakly Supervised Learning via Direct Loss Construction](https://mlsanity.com/paper/2212.06921/) (2022)
- [Semi-Supervised Text Classification via Self-Pretraining](https://mlsanity.com/paper/2109.15300/) (2021)
- [SimVQA: Exploring Simulated Environments for Visual Question Answering](https://mlsanity.com/paper/2203.17219/) (2022)
- [On the Importance of Calibration in Semi-supervised Learning](https://mlsanity.com/paper/2210.04783/) (2022)
- [Dual-Curriculum Teacher for Domain-Inconsistent Object Detection in Autonomous Driving](https://mlsanity.com/paper/2210.08748/) (2022)
- [Clue Me In: Semi-Supervised FGVC with Out-of-Distribution Data](https://mlsanity.com/paper/2112.02825/) (2021)
- [Taming Overconfident Prediction on Unlabeled Data from Hindsight](https://mlsanity.com/paper/2112.08200/) (2021)

## Abstract (arXiv)

In this paper we revisit the idea of pseudo-labeling in the context of semi-supervised learning where a learning algorithm has access to a small set of labeled samples and a large set of unlabeled samples. Pseudo-labeling works by applying pseudo-labels to samples in the unlabeled set by using a model trained on the combination of the labeled samples and any previously pseudo-labeled samples, and iteratively repeating this process in a self-training cycle. Current methods seem to have abandoned this approach in favor of consistency regularization methods that train models under a combination of different styles of self-supervised losses on the unlabeled samples and standard supervised losses on the labeled samples. We empirically demonstrate that pseudo-labeling can in fact be competitive with the state-of-the-art, while being more resilient to out-of-distribution samples in the unlabeled set. We identify two key factors that allow pseudo-labeling to achieve such remarkable results (1) applying curriculum learning principles and (2) avoiding concept drift by restarting model parameters before each self-training cycle. We obtain 94.91% accuracy on CIFAR-10 using only 4,000 labeled samples, and 68.87% top-1 accuracy on Imagenet-ILSVRC using only 10% of the labeled samples. The code is available at https://github.com/uvavision/Curriculum-Labeling