mlsanityData to 7 Oct 2026

Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach

arXiv 1609.03683 ↗First posted 13 Sep 2016stat.ML cs.LGcited by 663

Not on the map. Why?

Abstract (arXiv)

We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and multiplication, provided that we know the probability of each class being corrupted into another. We further show how one can estimate these probabilities, adapting a recent technique for noise estimation to the multi-class setting, and thus providing an end-to-end framework. Extensive experiments on MNIST, IMDB, CIFAR-10, CIFAR-100 and a large scale dataset of clothing images employing a diversity of architectures --- stacking dense, convolutional, pooling, dropout, batch normalization, word embedding, LSTM and residual layers --- demonstrate the noise robustness of our proposals. Incidentally, we also prove that, when ReLU is the only non-linearity, the loss curvature is immune to class-dependent label noise.

Key facts. Cited by 663 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 173; 2022: 138; 2023: 124; 2024: 89; 2025: 71; 2026: 68.

663 papers.

Sort:

By area:

  1. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  2. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
  3. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  4. 2024Provably Robust DPO: Aligning Language Models with Noisy Feedbackcited by 133
  5. 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
  6. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 112
  7. 2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
  8. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  9. 2021Provably End-to-end Label-Noise Learning without Anchor Pointscited by 97
  10. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  11. 2021Augmentation Strategies for Learning with Noisy Labelscited by 86
  12. 2022Self-Training: A Surveycited by 85
38 more
  1. 2021FINE Samples for Learning with Noisy Labelscited by 82
  2. 2021Breaking the Dilemma of Medical Image-to-image Translationcited by 82
  3. 2021Jo-SRC: A Contrastive Approach for Combating Noisy Labelscited by 80
  4. 2021Robust Audio-Visual Instance Discriminationcited by 80
  5. 2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
  6. 2022UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learningcited by 77
  7. 2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74
  8. 2021NRGNN: Learning a Label Noise-Resistant Graph Neural Network on Sparsely and Noisily Labeled Graphscited by 73
  9. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  10. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  11. 2021Learning Fast Sample Re-weighting Without Reward Datacited by 69
  12. 2022Learn From All: Erasing Attention Consistency for Noisy Label Facial Expression Recognitioncited by 67
  13. 2023Data Pruning via Moving-one-Sample-outcited by 67
  14. 2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65
  15. 2021Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularizationcited by 62
  16. 2022Part-based Pseudo Label Refinement for Unsupervised Person Re-identificationcited by 59
  17. 2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 58
  18. 2021NGC: A Unified Framework for Learning with Open-World Noisy Datacited by 58
  19. 2021Detecting Corrupted Labels Without Training a Model to Predictcited by 57
  20. 2021Exploring Memorization in Adversarial Trainingcited by 55
  21. 2022Learning with Neighbor Consistency for Noisy Labelscited by 55
  22. 2022Robust Cross-Modal Representation Learning with Progressive Self-Distillationcited by 55
  23. 2023Towards Label-free Scene Understanding by Vision Foundation Modelscited by 54
  24. 2021Co-learning: Learning from Noisy Labels with Self-supervisioncited by 50
  25. 2021Instance-dependent Label-noise Learning under a Structural Causal Modelcited by 50
  26. 2021Self-Training with Weak Supervisioncited by 49
  27. 2021Instance-Dependent Partial Label Learningcited by 49
  28. 2022Instance-Dependent Label-Noise Learning with Manifold-Regularized Transition Matrix Estimationcited by 49
  29. 2021From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimationcited by 48
  30. 2021Sample Selection for Fair and Robust Trainingcited by 48
  31. 2021Hard Sample Aware Noise Robust Learning for Histopathology Image Classificationcited by 48
  32. 2022ProMix: Combating Label Noise via Maximizing Clean Sample Utilitycited by 48
  33. 2021Asymmetric Loss Functions for Learning with Noisy Labelscited by 46
  34. 2022Investigating Why Contrastive Learning Benefits Robustness Against Label Noisecited by 46
  35. 2023When Source-Free Domain Adaptation Meets Learning with Noisy Labelscited by 46
  36. 2022Robust Contrastive Learning against Noisy Viewscited by 45
  37. 2021Active label cleaning for improved dataset quality under resource constraintscited by 44
  38. 2021MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentationcited by 43

50 most cited of 663 All 663 (JSON)