# Is your noise correction noisy? PLS: Robustness to label noise with two stage detection

arXiv: https://arxiv.org/abs/2210.04578 · HTML: https://mlsanity.com/paper/2210.04578/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 45 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise](https://mlsanity.com/paper/1802.05300/) (2018)
- [Probabilistic End-to-end Noise Correction for Learning with Noisy Labels](https://mlsanity.com/paper/1903.07788/) (2019)
- [Learning with Feature-Dependent Label Noise: A Progressive Approach](https://mlsanity.com/paper/2103.07756/) (2021)
- [Multi-Objective Interpolation Training for Robustness to Label Noise](https://mlsanity.com/paper/2012.04462/) (2020)
- [Jo-SRC: A Contrastive Approach for Combating Noisy Labels](https://mlsanity.com/paper/2103.13029/) (2021)
- [Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks](https://mlsanity.com/paper/1706.00038/) (2017)
- [Faster Meta Update Strategy for Noise-Robust Deep Learning](https://mlsanity.com/paper/2104.15092/) (2021)
- [Webly Supervised Fine-Grained Recognition: Benchmark Datasets and An Approach](https://mlsanity.com/paper/2108.02399/) (2021)
- [EvidentialMix: Learning with Combined Open-set and Closed-set Noisy Labels](https://mlsanity.com/paper/2011.05704/) (2020)
- [Learning Soft Labels via Meta Learning](https://mlsanity.com/paper/2009.09496/) (2020)
- [i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning](https://mlsanity.com/paper/2010.08887/) (2020)
- [PropMix: Hard Sample Filtering and Proportional MixUp for Learning with Noisy Labels](https://mlsanity.com/paper/2110.11809/) (2021)
- [Towards Robust Learning with Different Label Noise Distributions](https://mlsanity.com/paper/1912.08741/) (2019)

## Built on by (most cited first, 11 of 11)

- [Manifold DivideMix: A Semi-Supervised Contrastive Learning Framework for Severe Label Noise](https://mlsanity.com/paper/2308.06861/) (2023)
- [Label Noise: Ignorance Is Bliss](https://mlsanity.com/paper/2411.00079/) (2024)
- [Pi-DUAL: Using Privileged Information to Distinguish Clean from Noisy Labels](https://mlsanity.com/paper/2310.06600/) (2023)
- [Unleashing the Potential of Open-set Noisy Samples Against Label Noise for Medical Image Classification](https://mlsanity.com/paper/2406.12293/) (2024)
- [Open set label noise learning with robust sample selection and margin-guided module](https://mlsanity.com/paper/2501.04269/) (2025)
- [CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labels](https://mlsanity.com/paper/2605.23254/) (2026)
- [An accurate detection is not all you need to combat label noise in web-noisy datasets](https://mlsanity.com/paper/2407.05528/) (2024)
- [SSP-RACL: Classification of Noisy Fundus Images with Self-Supervised Pretraining and Robust Adaptive Credal Loss](https://mlsanity.com/paper/2409.18147/) (2024)
- [Debiased Orthogonal Boundary-Driven Efficient Noise Mitigation](https://mlsanity.com/paper/2410.01944/) (2024)
- [Distribution-aware Noisy-label Crack Segmentation](https://mlsanity.com/paper/2410.09409/) (2024)
- [Why Can Accurate Models Be Learned from Inaccurate Annotations?](https://mlsanity.com/paper/2505.16159/) (2025)

## Abstract (arXiv)

Designing robust algorithms capable of training accurate neural networks on uncurated datasets from the web has been the subject of much research as it reduces the need for time consuming human labor. The focus of many previous research contributions has been on the detection of different types of label noise; however, this paper proposes to improve the correction accuracy of noisy samples once they have been detected. In many state-of-the-art contributions, a two phase approach is adopted where the noisy samples are detected before guessing a corrected pseudo-label in a semi-supervised fashion. The guessed pseudo-labels are then used in the supervised objective without ensuring that the label guess is likely to be correct. This can lead to confirmation bias, which reduces the noise robustness. Here we propose the pseudo-loss, a simple metric that we find to be strongly correlated with pseudo-label correctness on noisy samples. Using the pseudo-loss, we dynamically down weight under-confident pseudo-labels throughout training to avoid confirmation bias and improve the network accuracy. We additionally propose to use a confidence guided contrastive objective that learns robust representation on an interpolated objective between class bound (supervised) for confidently corrected samples and unsupervised representation for under-confident label corrections. Experiments demonstrate the state-of-the-art performance of our Pseudo-Loss Selection (PLS) algorithm on a variety of benchmark datasets including curated data synthetically corrupted with in-distribution and out-of-distribution noise, and two real world web noise datasets. Our experiments are fully reproducible github.com/PaulAlbert31/SNCF