mlsanityData to 7 Oct 2026

Mitigating Memorization of Noisy Labels by Clipping the Model Prediction

arXiv 2212.04055 ↗First posted 8 Dec 2022cs.LG cs.AIcited by 40

Not on the map. Why?

Abstract (arXiv)

In the presence of noisy labels, designing robust loss functions is critical for securing the generalization performance of deep neural networks. Cross Entropy (CE) loss has been shown to be not robust to noisy labels due to its unboundedness. To alleviate this issue, existing works typically design specialized robust losses with the symmetric condition, which usually lead to the underfitting issue. In this paper, our key idea is to induce a loss bound at the logit level, thus universally enhancing the noise robustness of existing losses. Specifically, we propose logit clipping (LogitClip), which clamps the norm of the logit vector to ensure that it is upper bounded by a constant. In this manner, CE loss equipped with our LogitClip method is effectively bounded, mitigating the overfitting to examples with noisy labels. Moreover, we present theoretical analyses to certify the noise-tolerant ability of LogitClip. Extensive experiments show that LogitClip not only significantly improves the noise robustness of CE loss, but also broadly enhances the generalization performance of popular robust losses.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 63 of its 85 references that are in the map. First posted on arXiv 8 Dec 2022.

Builds on

63 of 85 references.

References · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  5. 2016Wide Residual Networkscited by 3,057
  6. 2016Deep Learning with Differential Privacycited by 2,682
  7. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  8. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  9. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  10. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  11. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  12. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
38 more
  1. 2017A Closer Look at Memorization in Deep Networkscited by 936
  2. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  3. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
  4. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  5. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  6. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  7. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  8. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  9. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  10. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
  11. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  12. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
  13. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
  14. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  15. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  16. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
  17. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  18. 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
  19. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
  20. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  21. 2017Decoupling "when to update" from "how to update"cited by 230
  22. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  23. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  24. 2020Does label smoothing mitigate label noise?cited by 204
  25. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  26. 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
  27. 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
  28. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 112
  29. 2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
  30. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  31. 2020A Second-Order Approach to Learning with Instance-Dependent Label Noisecited by 90
  32. 2020Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noisecited by 82
  33. 2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
  34. 2019AdaCliP: Adaptive Clipping for Private SGDcited by 77
  35. 2015Beyond Convexity: Stochastic Quasi-Convex Optimizationcited by 75
  36. 2020Error-Bounded Correction of Noisy Labelscited by 74
  37. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  38. 2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 18; 2025: 8; 2026: 8.

40 papers.

Sort:

By area:

  1. 2024ARGS: Alignment as Reward-Guided Searchcited by 99
  2. 2024Process Reward Model with Q-Value Rankingscited by 78
  3. 2023Sequence Length Independent Norm-Based Generalization Bounds for Transformerscited by 28
  4. 2023Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Modelscited by 26
  5. 2024Why is SAM Robust to Label Noise?cited by 21
  6. 2024Automatic Dataset Construction (ADC): Sample Collection, Data Curation, and Beyondcited by 18
  7. 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 15
  8. 2024A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?cited by 13
  9. 2023Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practicalcited by 10
  10. 2024SAP: Corrective Machine Unlearning with Scaled Activation Projection for Label Noise Robustnesscited by 10
  11. 2024Slight Corruption in Pre-training Data Makes Better Diffusion Modelscited by 9
  12. 2023Regularly Truncated M-estimators for Learning with Noisy Labelscited by 8
28 more
  1. 2024Tackling Noisy Labels with Network Parameter Additive Decompositioncited by 8
  2. 2024On the Learnability of Out-of-distribution Detectioncited by 8
  3. 2024Learning from Noisy Labels via Conditional Distributionally Robust Optimizationcited by 6
  4. 2025ε-Softmax: Approximating One-Hot Vectors for Mitigating Label Noisecited by 6
  5. 2024Selective Learning: Towards Robust Calibration with Dynamic Regularizationcited by 5
  6. 2023Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervisioncited by 4
  7. 2024Optimized Gradient Clipping for Noisy Label Learningcited by 4
  8. 2025Exploring the Noise Robustness of Online Conformal Predictioncited by 4
  9. 2023Improve Noise Tolerance of Robust Loss via Noise-Awarenesscited by 3
  10. 2025Fuzzy-aware Loss for Source-free Domain Adaptation in Visual Emotion Recognitioncited by 3
  11. 2025RCC-PFL: Robust Client Clustering under Noisy Labels in Personalized Federated Learningcited by 3
  12. 2025Robust Classification with Noisy Labels Based on Posterior Maximizationcited by 2
  13. 2026FB-NLL: A Feature-Based Approach to Tackle Noisy Labels in Personalized Federated Learningcited by 2
  14. 2024Sharpness-Aware Minimization for Evolutionary Feature Construction in Regressioncited by 1
  15. 2024Combating Semantic Contamination in Learning with Label Noisecited by 1
  16. 2026Batch Normalization Amplifies Memorization and Privacy Riskscited by 1
  17. 2024EchoAlign: Bridging Generative and Discriminative Learning under Noisy Labelscited by 0
  18. 2024Bayesian uncertainty-aware deep learning with noisy labels: Tackling annotation ambiguity in EEG seizure detectioncited by 0
  19. 2024Understanding SAM's Robustness to Noisy Labels through Gradient Down-weightingcited by 0
  20. 2025Set a Thief to Catch a Thief: Combating Label Noise through Noisy Meta Learningcited by 0
  21. 2025Learning to Forget with Information Divergence Reweighted Objectives for Noisy Labelscited by 0
  22. 2025Local K-Similarity Constraint for Federated Learning with Label Noisecited by 0
  23. 2026Noise-aware few-shot learning through bi-directional multi-view prompt alignmentcited by 0
  24. 2026Intrinsic Gradient Suppression for Label-Noise Prompt Tuning in Vision-Language Modelscited by 0
  25. 2026Unbiased Alignment for Large Language Models with Noisy Preferencescited by 0
  26. 2026Robust Losses from Univariate Base Functions for Noisy-Label Learningcited by 0
  27. 2026Leveraging Dissimilarity Invariance as a Robust Anchor for Learning with Noisy Labelscited by 0
  28. 2026Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learningcited by 0