mlsanityData to 14 Oct 2026

Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels

arXiv 1805.07836 ↗First posted 20 May 2018cs.LG cs.CV stat.MLcited by 997

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) have achieved tremendous success in a variety of applications across many disciplines. Yet, their superior performance comes with the expensive cost of requiring correctly annotated large-scale datasets. Moreover, due to DNNs' rich capacity, errors in training labels can hamper performance. To combat this problem, mean absolute error (MAE) has recently been proposed as a noise-robust alternative to the commonly-used categorical cross entropy (CCE) loss. However, as we show in this paper, MAE can perform poorly with DNNs and challenging datasets. Here, we present a theoretically grounded set of noise-robust loss functions that can be seen as a generalization of MAE and CCE. Proposed loss functions can be readily applied with any existing DNN architecture and algorithm, while yielding good performance in a wide range of noisy label scenarios. We report results from experiments conducted with CIFAR-10, CIFAR-100 and FASHION-MNIST datasets and synthetically generated noisy labels.

Key facts. Cited by 997 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 180; 2022: 185; 2023: 189; 2024: 199; 2025: 150; 2026: 94.

997 papers.

Sort:

By area:

  1. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 523
  2. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  3. 2021Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentationcited by 268
  4. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 256
  5. 2023Language is All a Graph Needscited by 220
  6. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 210
  7. 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 199
  8. 2022Medical Image Segmentation Review: The success of U-Netcited by 191
  9. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 172
  10. 2022Backdoor Defense via Decoupling the Training Processcited by 153
  11. 2022Pretrained Language Models for Text Generation: A Surveycited by 122
  12. 2021Deep Learning with Label Differential Privacycited by 117
38 more
  1. 2021Learning Debiased Representation via Disentangled Feature Augmentationcited by 116
  2. 2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
  3. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  4. 2021Provably End-to-end Label-Noise Learning without Anchor Pointscited by 97
  5. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  6. 2022Debiasing Graph Neural Networks via Learning Disentangled Causal Substructurecited by 83
  7. 2021FINE Samples for Learning with Noisy Labelscited by 82
  8. 2021Jo-SRC: A Contrastive Approach for Combating Noisy Labelscited by 80
  9. 2021Robust Audio-Visual Instance Discriminationcited by 80
  10. 2021A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentationcited by 79
  11. 2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
  12. 2022Are All Losses Created Equal: A Neural Collapse Perspectivecited by 78
  13. 2022UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learningcited by 77
  14. 2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74
  15. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  16. 2021BiaSwap: Removing dataset bias with bias-tailored swapping augmentationcited by 70
  17. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  18. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  19. 2021Learning Fast Sample Re-weighting Without Reward Datacited by 69
  20. 2022MPCFormer: fast, performant and private Transformer inference with MPCcited by 69
  21. 2023X-Pruner: eXplainable Pruning for Vision Transformerscited by 68
  22. 2022Learn From All: Erasing Attention Consistency for Noisy Label Facial Expression Recognitioncited by 67
  23. 2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65
  24. 2021DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Datacited by 64
  25. 2021Fairness via Representation Neutralizationcited by 63
  26. 2022PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functionscited by 63
  27. 2021Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularizationcited by 62
  28. 2022The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph Generationcited by 60
  29. 2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 59
  30. 2022Part-based Pseudo Label Refinement for Unsupervised Person Re-identificationcited by 59
  31. 2021Detecting Corrupted Labels Without Training a Model to Predictcited by 58
  32. 2022Transferability in Deep Learning: A Surveycited by 57
  33. 2021Exploring Memorization in Adversarial Trainingcited by 55
  34. 2022Robust Cross-Modal Representation Learning with Progressive Self-Distillationcited by 55
  35. 2021Co-learning: Learning from Noisy Labels with Self-supervisioncited by 51
  36. 2022Learning Debiased Classifier with Biased Committeecited by 51
  37. 2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 50
  38. 2023C-SFDA: A Curriculum Learning Aided Self-Training Framework for Efficient Source Free Domain Adaptationcited by 50

50 most cited of 997 All 997 (JSON)