mlsanityData to 7 Oct 2026

Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels

arXiv 2105.04522 ↗First posted 10 May 2021cs.LG cs.CV stat.MLcited by 74

Not on the map. Why?

Abstract (arXiv)

Prior works have found it beneficial to combine provably noise-robust loss functions e.g., mean absolute error (MAE) with standard categorical loss function e.g. cross entropy (CE) to improve their learnability. Here, we propose to use Jensen-Shannon divergence as a noise-robust loss function and show that it interestingly interpolate between CE and MAE with a controllable mixing parameter. Furthermore, we make a crucial observation that CE exhibit lower consistency around noisy data points. Based on this observation, we adopt a generalized version of the Jensen-Shannon divergence for multiple distributions to encourage consistency around data points. Using this loss function, we show state-of-the-art results on both synthetic (CIFAR), and real-world (e.g., WebVision) noise with varying noise rates.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Builds on 34 of its 52 references that are in the map. First posted on arXiv 10 May 2021.

Builds on

34 of 52 references.

References · 34

  1. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  2. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  4. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  5. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  6. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  7. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  8. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  9. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
  10. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  11. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  12. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
22 more
  1. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  2. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
  3. 2020Are we done with ImageNet?cited by 329
  4. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  5. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  6. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  7. 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
  8. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  9. 2020Does label smoothing mitigate label noise?cited by 204
  10. 2017CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noisecited by 181
  11. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  12. 2018Learning to Learn from Noisy Labeled Datacited by 160
  13. 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
  14. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  15. 2018Masking: A New Perspective of Noisy Supervisioncited by 93
  16. 2017Toward Robustness against Label Noise in Training Deep Discriminative Neural Networkscited by 78
  17. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  18. 2019On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract meanscited by 61
  19. 2017Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labelscited by 52
  20. 2020When Optimizing f-divergence is Robust with Label Noisecited by 44
  21. 2021Consistency Regularization for Adversarial Robustnesscited by 42
  22. 2019Graph convolutional networks for learning with few clean and many noisy labelscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 13; 2023: 14; 2024: 16; 2025: 16; 2026: 9.

74 papers.

Sort:

By area:

  1. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  2. 2022Learning with Neighbor Consistency for Noisy Labelscited by 55
  3. 2023When Source-Free Domain Adaptation Meets Learning with Noisy Labelscited by 46
  4. 2022Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guaranteescited by 34
  5. 2023Interpreting Pretrained Language Models via Concept Bottleneckscited by 33
  6. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationcited by 30
  7. 2023Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidancecited by 24
  8. 2021Consistency Regularization Can Improve Robustness to Label Noisecited by 18
  9. 2022NoisyMix: Boosting Model Robustness to Common Corruptionscited by 17
  10. 2022CTRL: Clustering Training Losses for Label Error Detectioncited by 17
  11. 2021Graph Kernel Neural Networkscited by 13
  12. 2023FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labelscited by 13
38 more
  1. 2021Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivitycited by 11
  2. 2023Generalized Semantic Segmentation by Self-Supervised Source Domain Projection and Multi-Level Contrastive Learningcited by 11
  3. 2022BoMD: Bag of Multi-label Descriptors for Noisy Chest X-ray Classificationcited by 9
  4. 2023The Representation Jensen-Shannon Divergencecited by 9
  5. 2024HPL-ESS: Hybrid Pseudo-Labeling for Unsupervised Event-based Semantic Segmentationcited by 9
  6. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMscited by 9
  7. 2023Dynamics-Aware Loss for Learning with Label Noisecited by 8
  8. 2024Learning Segmented 3D Gaussians via Efficient Feature Unprojection for Zero-shot Neural Scene Segmentationcited by 8
  9. 2022Certifying Out-of-Domain Generalization for Blackbox Functionscited by 7
  10. 2022Learning Algorithm Generalization Error Bounds via Auxiliary Distributionscited by 6
  11. 2022Dynamic Loss For Robust Learningcited by 6
  12. 2025ε-Softmax: Approximating One-Hot Vectors for Mitigating Label Noisecited by 6
  13. 2024Rethinking the impact of noisy labels in graph classification: A utility and privacy perspectivecited by 5
  14. 2025CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimizationcited by 5
  15. 2023Leveraging an Alignment Set in Tackling Instance-Dependent Label Noisecited by 4
  16. 2023Boundary-weighted logit consistency improves calibration of segmentation networkscited by 4
  17. 2024From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisionscited by 4
  18. 2024Optimized Gradient Clipping for Noisy Label Learningcited by 4
  19. 2025Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentationcited by 4
  20. 2025ChronoSelect: Robust Learning with Noisy Labels via Dynamics Temporal Memorycited by 4
  21. 2021Does Adversarial Oversampling Help us?cited by 3
  22. 2021CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learningcited by 3
  23. 2023Improve Noise Tolerance of Robust Loss via Noise-Awarenesscited by 3
  24. 2023Logistic-Normal Likelihoods for Heteroscedastic Label Noisecited by 3
  25. 2023Enhancing Noise-Robust Losses for Large-Scale Noisy Data Learningcited by 3
  26. 2024Indirectly Parameterized Concrete Autoencoderscited by 3
  27. 2024Robust Semi-supervised Learning via f-Divergence and α-R'enyi Divergencecited by 3
  28. 2025Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled yet Hard-to-Learn Samples in Noisy Datacited by 3
  29. 2022Split-PU: Hardness-aware Training Strategy for Positive-Unlabeled Learningcited by 2
  30. 2022SoftCTC -- Semi-Supervised Learning for Text Recognition using Soft Pseudo-Labelscited by 2
  31. 2023Smooth and Stepwise Self-Distillation for Object Detectioncited by 2
  32. 2024AlleNoise: large-scale text classification benchmark dataset with real-world label noisecited by 2
  33. 2025Text-to-3D Generation using Jensen-Shannon Score Distillationcited by 2
  34. 2025Introducing Fractional Classification Loss for Robust Learning with Noisy Labelscited by 2
  35. 2025Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scorescited by 2
  36. 2022Maximising the Utility of Validation Sets for Imbalanced Noisy-label Meta-learningcited by 1
  37. 2023Label Noise: Correcting the Forward-Correctioncited by 1
  38. 2024Dynamic Loss Decay based Robust Oriented Object Detection on Remote Sensing Images with Noisy Labelscited by 1

50 most cited of 74 All 74 (JSON)