# Label Noise: Ignorance Is Bliss

arXiv: https://arxiv.org/abs/2411.00079 · HTML: https://mlsanity.com/paper/2411.00079/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 56 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations](https://mlsanity.com/paper/2110.12088/) (2021)
- [Learning with Instance-Dependent Label Noise: A Sample Sieve Approach](https://mlsanity.com/paper/2010.02347/) (2020)
- [Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates](https://mlsanity.com/paper/1910.03231/) (2019)
- [Robust Training under Label Noise by Over-parameterization](https://mlsanity.com/paper/2202.14026/) (2022)
- [A Second-Order Approach to Learning with Instance-Dependent Label Noise](https://mlsanity.com/paper/2012.11854/) (2020)
- [Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels](https://mlsanity.com/paper/2103.13646/) (2021)
- [Classification with Asymmetric Label Noise: Consistency and Maximal Denoising](https://mlsanity.com/paper/1303.1208/) (2013)
- [Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier](https://mlsanity.com/paper/1906.02903/) (2019)
- [Making Risk Minimization Tolerant to Label Noise](https://mlsanity.com/paper/1403.3610/) (2014)
- [ProMix: Combating Label Noise via Maximizing Clean Sample Utility](https://mlsanity.com/paper/2207.10276/) (2022)
- [Investigating Why Contrastive Learning Benefits Robustness Against Label Noise](https://mlsanity.com/paper/2201.12498/) (2022)
- [Contrastive Learning Improves Model Robustness Under Label Noise](https://mlsanity.com/paper/2104.08984/) (2021)
- [Learning from Label Proportions by Learning with Label Noise](https://mlsanity.com/paper/2203.02496/) (2022)
- [A Generalized Neyman-Pearson Criterion for Optimal Domain Adaptation](https://mlsanity.com/paper/1810.01545/) (2018)
- [For self-supervised learning, Rationality implies generalization, provably](https://mlsanity.com/paper/2010.08508/) (2020)
- [A linear adjustment based approach to posterior drift in transfer learning](https://mlsanity.com/paper/2111.10841/) (2021)
- [Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Space](https://mlsanity.com/paper/2206.01106/) (2022)
- [Is your noise correction noisy? PLS: Robustness to label noise with two stage detection](https://mlsanity.com/paper/2210.04578/) (2022)
- [Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations](https://mlsanity.com/paper/2305.12715/) (2023)

## Built on by (most cited first, 9 of 9)

- [Domain Generalization Under Posterior Drift](https://mlsanity.com/paper/2510.04441/) (2025)
- [PAC Learnability in the Presence of Performativity](https://mlsanity.com/paper/2510.08335/) (2025)
- [Optimal Regularization for Performative Learning](https://mlsanity.com/paper/2510.12249/) (2025)
- [Calibrated Principal Component Regression](https://mlsanity.com/paper/2510.19020/) (2025)
- [Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noise](https://mlsanity.com/paper/2604.06468/) (2026)
- [Vertical Fusion: Condensing Internal Representations for Robust ViT Classification](https://mlsanity.com/paper/2607.10391/) (2026)
- [GERIS: A Game-Theoretic Framework for Filtering Instance-Dependent Label Noise in License Plate Data Augmentation](https://mlsanity.com/paper/2609.31731/) (2026)
- [Estimation of the Label-Noise Transition Matrix with Performance Guarantees via Selective Classification](https://mlsanity.com/paper/2609.39829/) (2026)
- [Retrieval-Based In-Context Learning: A Domain Adaptation Framework](https://mlsanity.com/paper/2610.05717/) (2026)

## Abstract (arXiv)

We establish a new theoretical framework for learning under multi-class, instance-dependent label noise. This framework casts learning with label noise as a form of domain adaptation, in particular, domain adaptation under posterior drift. We introduce the concept of relative signal strength (RSS), a pointwise measure that quantifies the transferability from noisy to clean posterior. Using RSS, we establish nearly matching upper and lower bounds on the excess risk. Our theoretical findings support the simple Noise Ignorant Empirical Risk Minimization (NI-ERM) principle, which minimizes empirical risk while ignoring label noise. Finally, we translate this theoretical insight into practice: by using NI-ERM to fit a linear classifier on top of a self-supervised feature extractor, we achieve state-of-the-art performance on the CIFAR-N data challenge.