Training Classifiers that are Universally Robust to All Label Noise Levels
Not on the map. Why?
Abstract (arXiv)
For classification tasks, deep neural networks are prone to overfitting in the presence of label noise. Although existing methods are able to alleviate this problem at low noise levels, they encounter significant performance reduction at high noise levels, or even at medium noise levels when the label noise is asymmetric. To train classifiers that are universally robust to all noise levels, and that are not sensitive to any variation in the noise model, we propose a distillation-based framework that incorporates a new subcategory of Positive-Unlabeled learning. In particular, we shall assume that a small subset of any given noisy dataset is known to have correct labels, which we treat as "positive", while the remaining noisy subset is treated as "unlabeled". Our framework consists of the following two components: (1) We shall generate, via iterative updates, an augmented clean subset with additional reliable "positive" samples filtered from "unlabeled" samples; (2) We shall train a teacher model on this larger augmented clean set. With the guidance of the teacher model, we then train a student model on the whole dataset. Experiments were conducted on the CIFAR-10 dataset with synthetic label noise at multiple noise levels for both symmetric and asymmetric noise. The results show that our framework generally outperforms at medium to high noise levels. We also evaluated our framework on Clothing1M, a real-world noisy dataset, and we achieved 2.94% improvement in accuracy over existing state-of-the-art methods.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 27 May 2021.
Builds on
27 of 43 references.
References · 27
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2016Identity Mappings in Deep Residual Networkscited by 1,960
- 2016Understanding deep learning requires rethinking generalizationcited by 1,657
- 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
- 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
- 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
- 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
- 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
- 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
15 more
- 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
- 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
- 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
- 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
- 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
- 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
- 2017CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noisecited by 181
- 2017Learning from Noisy Labels with Distillationcited by 160
- 2018Learning to Learn from Noisy Labeled Datacited by 160
- 2015A Light CNN for Deep Face Representation with Noisy Labelscited by 126
- 2019Deep Self-Learning From Noisy Labelscited by 122
- 2017Toward Robustness against Label Noise in Training Deep Discriminative Neural Networkscited by 78
- 2018Deep Bilevel Learningcited by 48
- 2017Learning Deep Networks from Noisy Labels with Dropout Regularizationcited by 32
- 2018A Semi-Supervised Two-Stage Approach to Learning from Noisy Labelscited by 29
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.