mlsanityData to 14 Oct 2026

Label Noise: Ignorance Is Bliss

arXiv 2411.00079 ↗First posted 31 Oct 2024cs.LG stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

We establish a new theoretical framework for learning under multi-class, instance-dependent label noise. This framework casts learning with label noise as a form of domain adaptation, in particular, domain adaptation under posterior drift. We introduce the concept of relative signal strength (RSS), a pointwise measure that quantifies the transferability from noisy to clean posterior. Using RSS, we establish nearly matching upper and lower bounds on the excess risk. Our theoretical findings support the simple Noise Ignorant Empirical Risk Minimization (NI-ERM) principle, which minimizes empirical risk while ignoring label noise. Finally, we translate this theoretical insight into practice: by using NI-ERM to fit a linear classifier on top of a self-supervised feature extractor, we achieve state-of-the-art performance on the CIFAR-N data challenge.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 31 of its 56 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

31 of 56 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  5. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  6. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  7. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
  8. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  9. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
  10. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 416
  11. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 413
  12. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 256
19 more
  1. 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
  2. 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
  3. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  4. 2020A Second-Order Approach to Learning with Instance-Dependent Label Noisecited by 90
  5. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  6. 2013Classification with Asymmetric Label Noise: Consistency and Maximal Denoisingcited by 68
  7. 2019Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifiercited by 65
  8. 2014Making Risk Minimization Tolerant to Label Noisecited by 54
  9. 2022ProMix: Combating Label Noise via Maximizing Clean Sample Utilitycited by 48
  10. 2022Investigating Why Contrastive Learning Benefits Robustness Against Label Noisecited by 46
  11. 2021Contrastive Learning Improves Model Robustness Under Label Noisecited by 36
  12. 2022Learning from Label Proportions by Learning with Label Noisecited by 30
  13. 2018A Generalized Neyman-Pearson Criterion for Optimal Domain Adaptationcited by 23
  14. 2020For self-supervised learning, Rationality implies generalization, provablycited by 22
  15. 2021A linear adjustment based approach to posterior drift in transfer learningcited by 17
  16. 2022Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Spacecited by 15
  17. 2022Is your noise correction noisy? PLS: Robustness to label noise with two stage detectioncited by 11
  18. 2023Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurationscited by 9
  19. 2023Binary Classification with Instance and Label Dependent Label Noisecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 5.

9 papers.

  1. 2025Domain Generalization Under Posterior Driftcited by 3
  2. 2025PAC Learnability in the Presence of Performativitycited by 1
  3. 2025Optimal Regularization for Performative Learningcited by 1
  4. 2025Calibrated Principal Component Regressioncited by 0
  5. 2026Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noisecited by 0
  6. 2026Vertical Fusion: Condensing Internal Representations for Robust ViT Classificationcited by 0
  7. 2026GERIS: A Game-Theoretic Framework for Filtering Instance-Dependent Label Noise in License Plate Data Augmentationcited by 0
  8. 2026Estimation of the Label-Noise Transition Matrix with Performance Guarantees via Selective Classificationcited by 0
  9. 2026Retrieval-Based In-Context Learning: A Domain Adaptation Frameworkcited by 0