Data to 7 Oct 2026

Normalized Loss Functions for Deep Learning with Noisy Labels

arXiv 2006.13554 ↗First posted 24 Jun 2020cs.LG cs.CV stat.MLcited by 257

Not on the map. Why?

Abstract (arXiv)

Robust loss functions are essential for training accurate deep neural networks (DNNs) in the presence of noisy (incorrect) labels. It has been shown that the commonly used Cross Entropy (CE) loss is not robust to noisy labels. Whilst new loss functions have been designed, they are only partially robust. In this paper, we theoretically show by applying a simple normalization that: any loss can be made robust to noisy labels. However, in practice, simply being robust is not sufficient for a loss function to train accurate DNNs. By investigating several robust loss functions, we find that they suffer from a problem of underfitting. To address this, we propose a framework to build robust loss functions called Active Passive Loss (APL). APL combines two robust loss functions that mutually boost each other. Experiments on benchmark datasets demonstrate that the family of new loss functions created by our APL framework can consistently outperform state-of-the-art methods by large margins, especially under large noise rates such as 60% or 80% incorrect labels.

Key facts. Cited by 257 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 48; 2022: 42; 2023: 56; 2024: 54; 2025: 34; 2026: 22.

257 papers.

Sort:

By area:

2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
2022Backdoor Defense via Decoupling the Training Processcited by 149
2021Deep Learning with Label Differential Privacycited by 113
2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 112
2022Robust Training under Label Noise by Over-parameterizationcited by 93
2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74
2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
2023Guiding Pseudo-labels with Uncertainty Estimation for Source-free Unsupervised Domain Adaptationcited by 63
2021Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularizationcited by 62
2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 58

38 more

2022Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCocited by 52
2021Co-learning: Learning from Noisy Labels with Self-supervisioncited by 50
2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 49
2022ProMix: Combating Label Noise via Maximizing Clean Sample Utilitycited by 48
2021Asymmetric Loss Functions for Learning with Noisy Labelscited by 46
2023When Source-Free Domain Adaptation Meets Learning with Noisy Labelscited by 46
2021Improved Regularization and Robustness for Fine-tuning in Neural Networkscited by 45
2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 43
2021LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environmentcited by 40
2022Mitigating Memorization of Noisy Labels by Clipping the Model Predictioncited by 40
2021Learning with Noisy Labels via Sparse Regularizationcited by 38
2021Contrastive Learning Improves Model Robustness Under Label Noisecited by 36
2021Joint Negative and Positive Learning for Noisy Labelscited by 34
2022Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guaranteescited by 34
2021Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Networkcited by 30
2022To Aggregate or Not? Learning with Separate Noisy Labelscited by 29
2022Robust Training of Graph Neural Networks via Noise Governancecited by 29
2023BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistencycited by 27
2023Understanding and Mitigating the Label Noise in Pre-training on Downstream Taskscited by 27
2020Robust Imitation Learning from Noisy Demonstrationscited by 26
2022CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learningcited by 24
2022SELC: Self-Ensemble Label Correction Improves Learning with Noisy Labelscited by 24
2021Understanding Instance-Level Label Noise: Disparate Impacts and Treatmentscited by 22
2022Noise-aware Learning from Web-crawled Image-Text Data for Image Captioningcited by 22
2023Cross-modal Active Complementary Learning with Self-refining Correspondencecited by 22
2022On Learning Contrastive Representations for Learning with Noisy Labelscited by 21
2022From Noisy Prediction to True Label: Noisy Prediction Calibration via Generative Modelcited by 21
2022When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration Methodcited by 21
2023Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels?cited by 21
2021Noise-robust Graph Learning by Estimating and Leveraging Pairwise Interactionscited by 20
2021Adaptive Sample Selection for Robust Learning under Label Noisecited by 20
2021Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrectioncited by 20
2022FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentationcited by 20
2022Towards Robust Adaptive Object Detection under Noisy Annotationscited by 20
2023Robust Data Pruning under Label Noise via Maximizing Re-labeling Accuracycited by 20
2022Identifying Hard Noise in Long-Tailed Sample Distributioncited by 19
2021Consistency Regularization Can Improve Robustness to Label Noisecited by 18
2021Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regressioncited by 18

50 most cited of 257 All 257 (JSON)