# Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise

arXiv: https://arxiv.org/abs/1802.05300 · HTML: https://mlsanity.com/paper/1802.05300/

## Key facts

- Cited by 216 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2018.

## Built on by (most cited first, 30 of 216)

- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations](https://mlsanity.com/paper/2110.12088/) (2021)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Selective-Supervised Contrastive Learning with Noisy Labels](https://mlsanity.com/paper/2203.04181/) (2022)
- [Robust Training under Label Noise by Over-parameterization](https://mlsanity.com/paper/2202.14026/) (2022)
- [FINE Samples for Learning with Noisy Labels](https://mlsanity.com/paper/2102.11628/) (2021)
- [Jo-SRC: A Contrastive Approach for Combating Noisy Labels](https://mlsanity.com/paper/2103.13029/) (2021)
- [Robust Audio-Visual Instance Discrimination](https://mlsanity.com/paper/2103.15916/) (2021)
- [Sample Selection with Uncertainty of Losses for Learning with Noisy Labels](https://mlsanity.com/paper/2106.00445/) (2021)
- [Improved Online Conformal Prediction via Strongly Adaptive Online Learning](https://mlsanity.com/paper/2302.07869/) (2023)
- [UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning](https://mlsanity.com/paper/2203.14542/) (2022)
- [Open-set Label Noise Can Improve Robustness Against Inherent Label Noise](https://mlsanity.com/paper/2106.10891/) (2021)
- [Part-based Pseudo Label Refinement for Unsupervised Person Re-identification](https://mlsanity.com/paper/2203.14675/) (2022)
- [Co-learning: Learning from Noisy Labels with Self-supervision](https://mlsanity.com/paper/2108.04063/) (2021)
- [Self-Training with Weak Supervision](https://mlsanity.com/paper/2104.05514/) (2021)
- [Instance-Dependent Label-Noise Learning with Manifold-Regularized Transition Matrix Estimation](https://mlsanity.com/paper/2206.02791/) (2022)
- [Hard Sample Aware Noise Robust Learning for Histopathology Image Classification](https://mlsanity.com/paper/2112.03694/) (2021)
- [ProMix: Combating Label Noise via Maximizing Clean Sample Utility](https://mlsanity.com/paper/2207.10276/) (2022)
- [Deep Learning for Suicide and Depression Identification with Unsupervised Label Correction](https://mlsanity.com/paper/2102.09427/) (2021)
- [MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2103.05254/) (2021)
- [On Tilted Losses in Machine Learning: Theory and Applications](https://mlsanity.com/paper/2109.06141/) (2021)
- [Mitigating Memorization of Noisy Labels by Clipping the Model Prediction](https://mlsanity.com/paper/2212.04055/) (2022)
- [Faster Meta Update Strategy for Noise-Robust Deep Learning](https://mlsanity.com/paper/2104.15092/) (2021)
- [Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterion](https://mlsanity.com/paper/2106.09291/) (2021)
- [Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning](https://mlsanity.com/paper/2211.03044/) (2022)
- [Joint Negative and Positive Learning for Noisy Labels](https://mlsanity.com/paper/2104.06574/) (2021)
- [Self-Filtering: A Noise-Aware Sample Selection for Label Noise with Confidence Penalization](https://mlsanity.com/paper/2208.11351/) (2022)
- [Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network](https://mlsanity.com/paper/2105.13001/) (2021)
- [Uncertainty-Aware Learning Against Label Noise on Imbalanced Datasets](https://mlsanity.com/paper/2207.05471/) (2022)

## Abstract (arXiv)

The growing importance of massive datasets used for deep learning makes robustness to label noise a critical property for classifiers to have. Sources of label noise include automatic labeling, non-expert labeling, and label corruption by data poisoning adversaries. Numerous previous works assume that no source of labels can be trusted. We relax this assumption and assume that a small subset of the training data is trusted. This enables substantial label corruption robustness performance gains. In addition, particularly severe label noise can be combated by using a set of trusted data with clean labels. We utilize trusted data by proposing a loss correction technique that utilizes trusted examples in a data-efficient manner to mitigate the effects of label noise on deep neural network classifiers. Across vision and natural language processing tasks, we experiment with various label noises at several strengths, and show that our method significantly outperforms existing methods.