# A Survey of Label-noise Representation Learning: Past, Present and Future

arXiv: https://arxiv.org/abs/2011.04406 · HTML: https://mlsanity.com/paper/2011.04406/

## Key facts

- Cited by 122 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2020.

## Built on by (most cited first, 30 of 122)

- [Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations](https://mlsanity.com/paper/2110.12088/) (2021)
- [Provably End-to-end Label-Noise Learning without Anchor Points](https://mlsanity.com/paper/2102.02400/) (2021)
- [MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluation](https://mlsanity.com/paper/2104.00773/) (2021)
- [Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels](https://mlsanity.com/paper/2102.05291/) (2021)
- [Understanding and Improving Graph Injection Attack by Promoting Unnoticeability](https://mlsanity.com/paper/2202.08057/) (2022)
- [FedCorr: Multi-Stage Federated Learning for Label Noise Correction](https://mlsanity.com/paper/2204.04677/) (2022)
- [Will Large-scale Generative Models Corrupt Future Datasets?](https://mlsanity.com/paper/2211.08095/) (2022)
- [Detecting Corrupted Labels Without Training a Model to Predict](https://mlsanity.com/paper/2110.06283/) (2021)
- [DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations](https://mlsanity.com/paper/2201.09637/) (2022)
- [ProMix: Combating Label Noise via Maximizing Clean Sample Utility](https://mlsanity.com/paper/2207.10276/) (2022)
- [Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions](https://mlsanity.com/paper/2111.08446/) (2021)
- [Impact of Preference Noise on the Alignment Performance of Generative Language Models](https://mlsanity.com/paper/2404.09824/) (2024)
- [Calibrated One-class Classification for Unsupervised Time Series Anomaly Detection](https://mlsanity.com/paper/2207.12201/) (2022)
- [Learning with Noisy Labels via Sparse Regularization](https://mlsanity.com/paper/2108.00192/) (2021)
- [Unsupervised Domain Adaptation of Black-Box Source Models](https://mlsanity.com/paper/2101.02839/) (2021)
- [To Aggregate or Not? Learning with Separate Noisy Labels](https://mlsanity.com/paper/2206.07181/) (2022)
- [Beyond Images: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features](https://mlsanity.com/paper/2202.01273/) (2022)
- [Mitigating Memorization of Noisy Labels via Regularization between Representations](https://mlsanity.com/paper/2110.09022/) (2021)
- [CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning](https://mlsanity.com/paper/2202.05613/) (2022)
- [RIM: Reliable Influence-based Active Learning on Graphs](https://mlsanity.com/paper/2110.14854/) (2021)
- [Noise-robust Graph Learning by Estimating and Leveraging Pairwise Interactions](https://mlsanity.com/paper/2106.07451/) (2021)
- [PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning](https://mlsanity.com/paper/2201.08984/) (2022)
- [A Survey on the Robustness of Computer Vision Models against Common Corruptions](https://mlsanity.com/paper/2305.06024/) (2023)
- [Early Stopping Against Label Noise Without Validation Data](https://mlsanity.com/paper/2502.07551/) (2025)
- [On the Robustness of Average Losses for Partial-Label Learning](https://mlsanity.com/paper/2106.06152/) (2021)
- [Learning From Long-Tailed Data With Noisy Labels](https://mlsanity.com/paper/2108.11096/) (2021)
- [Label-Noise Learning with Intrinsically Long-Tailed Data](https://mlsanity.com/paper/2208.09833/) (2022)
- [Late Stopping: Avoiding Confidently Learning from Mislabeled Examples](https://mlsanity.com/paper/2308.13862/) (2023)
- [On the Power-Law Hessian Spectrums in Deep Learning](https://mlsanity.com/paper/2201.13011/) (2022)
- [A Review of Pseudo-Labeling for Computer Vision](https://mlsanity.com/paper/2408.07221/) (2024)

## Abstract (arXiv)

Classical machine learning implicitly assumes that labels of the training data are sampled from a clean distribution, which can be too restrictive for real-world scenarios. However, statistical-learning-based methods may not train deep learning models robustly with these noisy labels. Therefore, it is urgent to design Label-Noise Representation Learning (LNRL) methods for robustly training deep models with noisy labels. To fully understand LNRL, we conduct a survey study. We first clarify a formal definition for LNRL from the perspective of machine learning. Then, via the lens of learning theory and empirical study, we figure out why noisy labels affect deep models' performance. Based on the theoretical guidance, we categorize different LNRL methods into three directions. Under this unified taxonomy, we provide a thorough discussion of the pros and cons of different categories. More importantly, we summarize the essential components of robust LNRL, which can spark new directions. Lastly, we propose possible research directions within LNRL, such as new datasets, instance-dependent LNRL, and adversarial LNRL. We also envision potential directions beyond LNRL, such as learning with feature-noise, preference-noise, domain-noise, similarity-noise, graph-noise and demonstration-noise.