# A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels

arXiv: https://arxiv.org/abs/1802.02679 · HTML: https://mlsanity.com/paper/1802.02679/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2018.

## Built on by (most cited first, 29 of 29)

- [Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling](https://mlsanity.com/paper/2111.12698/) (2021)
- [UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning](https://mlsanity.com/paper/2203.14542/) (2022)
- [Learning Fast Sample Re-weighting Without Reward Data](https://mlsanity.com/paper/2109.03216/) (2021)
- [Co-learning: Learning from Noisy Labels with Self-supervision](https://mlsanity.com/paper/2108.04063/) (2021)
- [LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environment](https://mlsanity.com/paper/2103.04173/) (2021)
- [Joint Negative and Positive Learning for Noisy Labels](https://mlsanity.com/paper/2104.06574/) (2021)
- [Boosting Co-teaching with Compression Regularization for Label Noise](https://mlsanity.com/paper/2104.13766/) (2021)
- [Robust Medical Image Classification from Noisy Labeled Data with Global and Local Representation Guided Co-training](https://mlsanity.com/paper/2205.04723/) (2022)
- [Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformer](https://mlsanity.com/paper/2207.11971/) (2022)
- [Two Wrongs Don't Make a Right: Combating Confirmation Bias in Learning with Label Noise](https://mlsanity.com/paper/2112.02960/) (2021)
- [PropMix: Hard Sample Filtering and Proportional MixUp for Learning with Noisy Labels](https://mlsanity.com/paper/2110.11809/) (2021)
- [Addressing out-of-distribution label noise in webly-labelled data](https://mlsanity.com/paper/2110.13699/) (2021)
- [ScanMix: Learning from Severe Label Noise via Semantic Clustering and Semi-Supervised Learning](https://mlsanity.com/paper/2103.11395/) (2021)
- [Label Cleaning Multiple Instance Learning: Refining Coarse Annotations on Single Whole-Slide Images](https://mlsanity.com/paper/2109.10778/) (2021)
- [On the Robustness of Monte Carlo Dropout Trained with Noisy Labels](https://mlsanity.com/paper/2103.12002/) (2021)
- [Semantic Clustering based Deduction Learning for Image Recognition and Classification](https://mlsanity.com/paper/2112.13165/) (2021)
- [SelfMix: Robust Learning Against Textual Label Noise with Self-Mixup Training](https://mlsanity.com/paper/2210.04525/) (2022)
- [SURE: SUrvey REcipes for building reliable and robust deep networks](https://mlsanity.com/paper/2403.00543/) (2024)
- [Compressing Features for Learning with Noisy Labels](https://mlsanity.com/paper/2206.13140/) (2022)
- [SplitNet: Learnable Clean-Noisy Label Splitting for Learning with Noisy Labels](https://mlsanity.com/paper/2211.11753/) (2022)
- [Imbalanced Medical Image Segmentation with Pixel-dependent Noisy Labels](https://mlsanity.com/paper/2501.06678/) (2025)
- [Co-matching: Combating Noisy Labels by Augmentation Anchoring](https://mlsanity.com/paper/2103.12814/) (2021)
- [Industry Scale Semi-Supervised Learning for Natural Language Understanding](https://mlsanity.com/paper/2103.15871/) (2021)
- [Boosting Semi-Supervised Face Recognition with Noise Robustness](https://mlsanity.com/paper/2105.04431/) (2021)
- [Training Classifiers that are Universally Robust to All Label Noise Levels](https://mlsanity.com/paper/2105.13892/) (2021)
- [DST: Data Selection and joint Training for Learning with Noisy Labels](https://mlsanity.com/paper/2103.00813/) (2021)
- [Generation and Analysis of Feature-Dependent Pseudo Noise for Training Deep Neural Networks](https://mlsanity.com/paper/2105.10796/) (2021)
- [Deep Learning with Label Noise: A Hierarchical Approach](https://mlsanity.com/paper/2205.14299/) (2022)
- [Class Prototype-based Cleaner for Label Noise Learning](https://mlsanity.com/paper/2212.10766/) (2022)

## Abstract (arXiv)

The recent success of deep neural networks is powered in part by large-scale well-labeled training data. However, it is a daunting task to laboriously annotate an ImageNet-like dateset. On the contrary, it is fairly convenient, fast, and cheap to collect training images from the Web along with their noisy labels. This signifies the need of alternative approaches to training deep neural networks using such noisy labels. Existing methods tackling this problem either try to identify and correct the wrong labels or reweigh the data terms in the loss function according to the inferred noisy rates. Both strategies inevitably incur errors for some of the data points. In this paper, we contend that it is actually better to ignore the labels of some of the data points than to keep them if the labels are incorrect, especially when the noisy rate is high. After all, the wrong labels could mislead a neural network to a bad local optimum. We suggest a two-stage framework for the learning from noisy labels. In the first stage, we identify a small portion of images from the noisy training set of which the labels are correct with a high probability. The noisy labels of the other images are ignored. In the second stage, we train a deep neural network in a semi-supervised manner. This framework effectively takes advantage of the whole training set and yet only a portion of its labels that are most likely correct. Experiments on three datasets verify the effectiveness of our approach especially when the noisy rate is high.