Data to 7 Oct 2026

Does label smoothing mitigate label noise?

arXiv 2003.02819 ↗First posted 5 Mar 2020cs.LG stat.MLcited by 204

Not on the map. Why?

Abstract (arXiv)

Label smoothing is commonly used in training deep learning models, wherein one-hot training labels are mixed with uniform label vectors. Empirically, smoothing has been shown to improve both predictive performance and model calibration. In this paper, we study whether label smoothing is also effective as a means of coping with label noise. While label smoothing apparently amplifies this problem --- being equivalent to injecting symmetric noise to the labels --- we show how it relates to a general family of loss-correction techniques from the label noise literature. Building on this connection, we show that label smoothing is competitive with loss-correction under label noise. Further, we show that when distilling models from noisy data, label smoothing of the teacher is beneficial; this is in contrast to recent findings for noise-free problems, and sheds further light on settings where label smoothing is beneficial.

Key facts. Cited by 204 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 38; 2022: 48; 2023: 44; 2024: 37; 2025: 26; 2026: 11.

204 papers.

Sort:

By area:

2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 151
2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 131
2021Deep Learning with Label Differential Privacycited by 113
2021SPICE: Semantic Pseudo-labeling for Image Clusteringcited by 104
2022Robust Training under Label Noise by Over-parameterizationcited by 93
2021The Devil is in the Margin: Margin-based Label Smoothing for Network Calibrationcited by 91
2021FINE Samples for Learning with Noisy Labelscited by 82
2021Multi-Label Learning from Single Positive Labelscited by 81
2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74

38 more

2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
2021Learning Fast Sample Re-weighting Without Reward Datacited by 69
2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65
2022Part-based Pseudo Label Refinement for Unsupervised Person Re-identificationcited by 59
2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 58
2022Learning with Neighbor Consistency for Noisy Labelscited by 55
2022DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotationscited by 54
2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detectioncited by 54
2021Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Studycited by 50
2021Active label cleaning for improved dataset quality under resource constraintscited by 44
2022Mitigating Memorization of Noisy Labels by Clipping the Model Predictioncited by 40
2021Incremental Learning in Semantic Segmentation from Image Labelscited by 39
2023Re-thinking Model Inversion Attacks Against Deep Neural Networkscited by 39
2022Large Loss Matters in Weakly Supervised Multi-Label Classificationcited by 38
2022Threshold Matters in WSSS: Manipulating the Activation for the Robust and Accurate Segmentation Model Against Thresholdscited by 37
2022Towards Robust Ranker for Text Retrievalcited by 37
2024RIME: Robust Preference-based Reinforcement Learning with Noisy Preferencescited by 37
2022Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guaranteescited by 34
2022Self-Filtering: A Noise-Aware Sample Selection for Label Noise with Confidence Penalizationcited by 33
2023FedNoRo: Towards Noise-Robust Federated Learning by Addressing Class Imbalance and Label Noise Heterogeneitycited by 33
2022Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing?cited by 32
2022To Aggregate or Not? Learning with Separate Noisy Labelscited by 29
2021Continual Learning on Noisy Data Streams via Self-Purified Replaycited by 27
2022Beyond Images: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Featurescited by 27
2023From Shortcuts to Triggers: Backdoor Defense with Denoised PoEcited by 26
2023Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Modelscited by 26
2025GRAM: A Generative Foundation Reward Model for Reward Generalizationcited by 25
2022CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learningcited by 24
2021Understanding Instance-Level Label Noise: Disparate Impacts and Treatmentscited by 22
2021Efficient Algorithms for Learning from Coarse Labelscited by 21
2022Is BERT Robust to Label Noise? A Study on Learning with Noisy Labels in Text Classificationcited by 21
2022Boundary Smoothing for Named Entity Recognitioncited by 21
2022From Noisy Prediction to True Label: Noisy Prediction Calibration via Generative Modelcited by 21
2021Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrectioncited by 20
2022NLIP: Noise-robust Language-Image Pre-trainingcited by 20
2023Understanding Self-Distillation in the Presence of Label Noisecited by 20
2023Novel Class Discovery for Long-tailed Recognitioncited by 19
2021Consistency Regularization Can Improve Robustness to Label Noisecited by 18

50 most cited of 204 All 204 (JSON)