mlsanityData to 7 Oct 2026

Robust Loss Functions under Label Noise for Deep Neural Networks

arXiv 1712.09482 ↗First posted 27 Dec 2017stat.ML cs.LGcited by 441

Not on the map. Why?

Abstract (arXiv)

In many applications of classifier learning, training data suffers from label noise. Deep networks are learned using huge training data where the problem of noisy labels is particularly relevant. The current techniques proposed for learning deep networks under label noise focus on modifying the network architecture and on algorithms for estimating true labels from noisy labels. An alternate approach would be to look for loss functions that are inherently noise-tolerant. For binary classification there exist theoretical results on loss functions that are robust to label noise. In this paper, we provide some sufficient conditions on a loss function so that risk minimization under that loss function would be inherently tolerant to label noise for multiclass classification problems. These results generalize the existing results on noise-tolerant loss functions for binary classification. We study some of the widely used loss functions in deep networks and show that the loss function based on mean absolute value of error is inherently robust to label noise. Thus standard back propagation is enough to learn the true classifier even under label noise. Through experiments, we illustrate the robustness of risk minimization with such loss functions for learning neural networks.

Key facts. Cited by 441 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 95; 2022: 88; 2023: 77; 2024: 87; 2025: 56; 2026: 38.

441 papers.

Sort:

By area:

  1. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
  2. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  3. 2024WARM: On the Benefits of Weight Averaged Reward Modelscited by 145
  4. 2022The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformerscited by 121
  5. 2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
  6. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  7. 2023The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecastingcited by 100
  8. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  9. 2021FINE Samples for Learning with Noisy Labelscited by 82
  10. 2021Robust Audio-Visual Instance Discriminationcited by 80
  11. 2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74
  12. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
38 more
  1. 2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65
  2. 2023Guiding Pseudo-labels with Uncertainty Estimation for Source-free Unsupervised Domain Adaptationcited by 63
  3. 2021Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularizationcited by 62
  4. 2022PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functionscited by 62
  5. 2022Part-based Pseudo Label Refinement for Unsupervised Person Re-identificationcited by 59
  6. 2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 58
  7. 2021Detecting Corrupted Labels Without Training a Model to Predictcited by 57
  8. 2022Learning with Neighbor Consistency for Noisy Labelscited by 55
  9. 2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 49
  10. 2021From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimationcited by 48
  11. 2021Hard Sample Aware Noise Robust Learning for Histopathology Image Classificationcited by 48
  12. 2022ProMix: Combating Label Noise via Maximizing Clean Sample Utilitycited by 48
  13. 2021Asymmetric Loss Functions for Learning with Noisy Labelscited by 46
  14. 2022Investigating Why Contrastive Learning Benefits Robustness Against Label Noisecited by 46
  15. 2023When Source-Free Domain Adaptation Meets Learning with Noisy Labelscited by 46
  16. 2022Robust Contrastive Learning against Noisy Viewscited by 45
  17. 2021Orthogonal Projection Losscited by 44
  18. 2021Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Trainingcited by 41
  19. 2022Mitigating Memorization of Noisy Labels by Clipping the Model Predictioncited by 40
  20. 2021Learning with Noisy Labels via Sparse Regularizationcited by 38
  21. 2024RIME: Robust Preference-based Reinforcement Learning with Noisy Preferencescited by 37
  22. 2021Contrastive Learning Improves Model Robustness Under Label Noisecited by 36
  23. 2021If your data distribution shifts, use self-learningcited by 36
  24. 2021Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterioncited by 36
  25. 2023Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithmscited by 36
  26. 2021Joint Negative and Positive Learning for Noisy Labelscited by 34
  27. 2022Self-Filtering: A Noise-Aware Sample Selection for Label Noise with Confidence Penalizationcited by 33
  28. 2023A Path to Simpler Models Starts With Noisecited by 33
  29. 2021TRACER: Extreme Attention Guided Salient Object Tracing Networkcited by 32
  30. 2023Theoretically Grounded Loss Functions and Algorithms for Score-Based Multi-Class Abstentioncited by 32
  31. 2021Joint Noise-Tolerant Learning and Meta Camera Shift Adaptation for Unsupervised Person Re-Identificationcited by 29
  32. 2022Robust Training of Graph Neural Networks via Noise Governancecited by 29
  33. 2021Uncertainty Estimation via Response Scaling for Pseudo-mask Noise Mitigation in Weakly-supervised Semantic Segmentationcited by 28
  34. 2022Uncertainty-Aware Learning Against Label Noise on Imbalanced Datasetscited by 28
  35. 2023Twin Contrastive Learning with Noisy Labelscited by 28
  36. 2023Understanding and Mitigating the Label Noise in Pre-training on Downstream Taskscited by 27
  37. 2021Mitigating Memorization of Noisy Labels via Regularization between Representationscited by 26
  38. 2024Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defercited by 26

50 most cited of 441 All 441 (JSON)