mlsanityData to 14 Oct 2026

Delving Deep into Label Smoothing

arXiv 2011.12562 ↗First posted 25 Nov 2020cs.CVcited by 94

Not on the map. Why?

Abstract (arXiv)

Label smoothing is an effective regularization tool for deep neural networks (DNNs), which generates soft labels by applying a weighted average between the uniform distribution and the hard label. It is often used to reduce the overfitting problem of training DNNs and further improve classification performance. In this paper, we aim to investigate how to generate more reliable soft labels. We present an Online Label Smoothing (OLS) strategy, which generates soft labels based on the statistics of the model prediction for the target category. The proposed OLS constructs a more reasonable probability distribution between the target categories and non-target categories to supervise DNNs. Experiments demonstrate that based on the same classification models, the proposed approach can effectively improve the classification performance on CIFAR-100, ImageNet, and fine-grained datasets. Additionally, the proposed method can significantly improve the robustness of DNN models to noisy labels compared to current label smoothing approaches.

Key facts. Cited by 94 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 25; 2023: 14; 2024: 22; 2025: 17; 2026: 5.

94 papers.

Sort:

By area:

  1. 2022Representation Compensation Networks for Continual Semantic Segmentationcited by 72
  2. 2023From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labelscited by 69
  3. 2022Eliciting and Learning with Soft Labels from Every Annotatorcited by 43
  4. 2023Towards Calibrated Robust Fine-Tuning of Vision-Language Modelscited by 35
  5. 2022Self-Distillation from the Last Mini-Batch for Consistency Regularizationcited by 29
  6. 2021Salient Objects in Cluttercited by 19
  7. 2022Relieving Long-tailed Instance Segmentation via Pairwise Class Balancecited by 18
  8. 2022ZLPR: A Novel Loss for Multi-label Classificationcited by 16
  9. 2024SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Modelscited by 16
  10. 2021Speechformer: Reducing Information Loss in Direct Speech Translationcited by 15
  11. 2022Event Transformer+. A multi-purpose solution for efficient event data processingcited by 13
  12. 2022Efficient One Pass Self-distillation with Zipf's Label Smoothingcited by 12
38 more
  1. 2022Soft Augmentation for Image Classificationcited by 11
  2. 2022BoMD: Bag of Multi-label Descriptors for Noisy Chest X-ray Classificationcited by 9
  3. 2023It Is All About Data: A Survey on the Effects of Data on Adversarial Robustnesscited by 9
  4. 2024AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labelscited by 9
  5. 2022Open-set Text Recognition via Character-Context Decouplingcited by 8
  6. 2023Regroup Median Loss for Combating Label Noisecited by 8
  7. 2024Noisy Label Processing for Classification: A Surveycited by 8
  8. 2023DynamicKD: An Effective Knowledge Distillation via Dynamic Entropy Correction-Based Distillation for Gap Optimizingcited by 7
  9. 2021Navigation-Oriented Scene Understanding for Robotic Autonomy: Learning to Segment Driveability in Egocentric Imagescited by 6
  10. 2022Inverse Image Frequency for Long-tailed Image Recognitioncited by 6
  11. 2022LUMix: Improving Mixup by Better Modelling Label Uncertaintycited by 6
  12. 2023Data Optimization in Deep Learning: A Surveycited by 6
  13. 2022Convolutional Fine-Grained Classification with Self-Supervised Target Relation Regularizationcited by 5
  14. 2022In and Out-of-Domain Text Adversarial Robustness via Label Smoothingcited by 5
  15. 2021Structure-Aware Label Smoothing for Graph Neural Networkscited by 4
  16. 2024Data-Driven Estimation of the False Positive Rate of the Bayes Binary Classifier via Soft Labelscited by 4
  17. 2025MaxSup: Overcoming Representation Collapse in Label Smoothingcited by 4
  18. 2021Jitter: Random Jittering Loss Functioncited by 3
  19. 2023Model Calibration in Dense Classification with Adaptive Label Perturbationcited by 3
  20. 2021Compensation Learningcited by 2
  21. 2022Combating Noisy Labels in Long-Tailed Image Classificationcited by 2
  22. 2024A Benchmark Dataset for Tornado Detection and Prediction using Full-Resolution Polarimetric Weather Radar Datacited by 2
  23. 2024Robust and Efficient Adversarial Defense in SNNs via Image Purification and Joint Detectioncited by 2
  24. 2024Adaptive Label Smoothing for Out-of-Distribution Detectioncited by 2
  25. 2024LLMD: A Large Language Model for Interpreting Longitudinal Medical Recordscited by 2
  26. 2025AG-VPReID 2025: Aerial-Ground Video-based Person Re-identification Challenge Resultscited by 2
  27. 2026Can LLMs Learn to Reason Robustly under Noisy Supervision?cited by 2
  28. 2021Multi-loss ensemble deep learning for chest X-ray classificationcited by 1
  29. 2021A Robust Initialization of Residual Blocks for Effective ResNet Training without Batch Normalizationcited by 1
  30. 2022Reprint: a randomized extrapolation based on principal components for data augmentationcited by 1
  31. 2022Going Beyond One-Hot Encoding in Classification: Can Human Uncertainty Improve Model Performance?cited by 1
  32. 2022Improving the Robustness and Generalization of Deep Neural Network with Confidence Threshold Reductioncited by 1
  33. 2022Solutions for Fine-grained and Long-tailed Snake Species Recognition in SnakeCLEF 2022cited by 1
  34. 2022Object Detection for Autonomous Dozerscited by 1
  35. 2022Soft-labeling Strategies for Rapid Sub-Typingcited by 1
  36. 2022Confidence-Aware Paced-Curriculum Learning by Label Smoothing for Surgical Scene Understandingcited by 1
  37. 2023From CAD models to soft point cloud labels: An automatic annotation pipeline for cheaply supervised 3D semantic segmentationcited by 1
  38. 2023Lightweight Self-Knowledge Distillation with Multi-source Information Fusioncited by 1

50 most cited of 94 All 94 (JSON)