mlsanityData to 7 Oct 2026

When Optimizing f-divergence is Robust with Label Noise

arXiv 2011.03687 ↗First posted 7 Nov 2020cs.LG stat.MLcited by 44

Not on the map. Why?

Abstract (arXiv)

We show when maximizing a properly defined f-divergence measure with respect to a classifier's predictions and the supervised labels is robust with label noise. Leveraging its variational form, we derive a nice decoupling property for a family of f-divergence measures when label noise presents, where the divergence is shown to be a linear combination of the variational difference defined on the clean distribution and a bias term introduced due to the noise. The above derivation helps us analyze the robustness of different f-divergence functions. With established robustness, this family of f-divergence functions arises as useful metrics for the problem of learning with noisy labels, which do not require the specification of the labels' noise rate. When they are possibly not robust, we propose fixes to make them so. In addition to the analytical results, we present thorough experimental evidence. Our code is available at https://github.com/UCSC-REAL/Robust-f-divergence-measures.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 9; 2023: 8; 2024: 5; 2025: 8; 2026: 2.

44 papers.

Sort:

By area:

  1. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
  2. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  3. 2022Are All Losses Created Equal: A Neural Collapse Perspectivecited by 78
  4. 2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74
  5. 2024LLM Unlearning via Loss Adjustment with Only Forget Datacited by 71
  6. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  7. 2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65
  8. 2021Detecting Corrupted Labels Without Training a Model to Predictcited by 57
  9. 2022Mitigating Memorization of Noisy Labels by Clipping the Model Predictioncited by 40
  10. 2024Measuring and Reducing LLM Hallucination without Gold-Standard Answerscited by 31
  11. 2022To Aggregate or Not? Learning with Separate Noisy Labelscited by 29
  12. 2021The Rich Get Richer: Disparate Impact of Semi-Supervised Learningcited by 28
32 more
  1. 2022Beyond Images: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Featurescited by 27
  2. 2021Mitigating Memorization of Noisy Labels via Regularization between Representationscited by 26
  3. 2022When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration Methodcited by 21
  4. 2021Consistency Regularization Can Improve Robustness to Label Noisecited by 18
  5. 2024Automatic Dataset Construction (ADC): Sample Collection, Data Curation, and Beyondcited by 18
  6. 2022Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributescited by 17
  7. 2023Distributionally Robust Post-hoc Classifiers under Prior Shiftscited by 16
  8. 2023Internally Rewarded Reinforcement Learningcited by 14
  9. 2021Can Less be More? When Increasing-to-Balancing Label Noise Rates Considered Beneficialcited by 13
  10. 2023Mitigating Label Noise through Data Ambiguationcited by 10
  11. 2023The Importance of Human-Labeled Data in the Era of LLMscited by 8
  12. 2024f-Divergence Based Classification: Beyond the Use of Cross-Entropycited by 8
  13. 2025ε-Softmax: Approximating One-Hot Vectors for Mitigating Label Noisecited by 6
  14. 2022SplitNet: Learnable Clean-Noisy Label Splitting for Learning with Noisy Labelscited by 5
  15. 2023Fairness Improves Learning from Noisily Labeled Long-Tailed Datacited by 5
  16. 2025When VLMs Meet Image Classification: Test Sets Renovation via Missing Label Identificationcited by 5
  17. 2025SelectMix: Enhancing Label Noise Robustness through Targeted Sample Mixingcited by 5
  18. 2021DuelGAN: A Duel Between Two Discriminators Stabilizes the GAN Trainingcited by 4
  19. 2021Rethinking Noisy Label Models: Labeler-Dependent Noise with Adversarial Awarenesscited by 4
  20. 2023Logistic-Normal Likelihoods for Heteroscedastic Label Noisecited by 3
  21. 2024Robust Semi-supervised Learning via f-Divergence and α-R'enyi Divergencecited by 3
  22. 2025Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled yet Hard-to-Learn Samples in Noisy Datacited by 3
  23. 2022Tackling Instance-Dependent Label Noise with Dynamic Distribution Calibrationcited by 2
  24. 2023Smooth and Stepwise Self-Distillation for Object Detectioncited by 2
  25. 2025Robust Classification with Noisy Labels Based on Posterior Maximizationcited by 2
  26. 2025A Unified Framework for Diffusion Model Unlearning with f-Divergencecited by 2
  27. 2023Bridging Generative and Discriminative Noisy-Label Learning via Direction-Agnostic EM Formulationcited by 1
  28. 2025A Dual Optimization View to Empirical Risk Minimization with f-Divergence Regularizationcited by 1
  29. 2021A Symmetric Loss Perspective of Reliable Machine Learningcited by 0
  30. 2025Learning to Forget with Information Divergence Reweighted Objectives for Noisy Labelscited by 0
  31. 2026Empirical Risk Minimization with f-Divergence Regularizationcited by 0
  32. 2026Explaining f-Divergence-Based Regularization via Local Curvature and Sharpness-Aware Minimizationcited by 0