mlsanityData to 13 Oct 2026

Learning from Noisy Labels via Dynamic Loss Thresholding

arXiv 2104.02570 ↗First posted 1 Apr 2021cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Numerous researches have proved that deep neural networks (DNNs) can fit everything in the end even given data with noisy labels, and result in poor generalization performance. However, recent studies suggest that DNNs tend to gradually memorize the data, moving from correct data to mislabeled data. Inspired by this finding, we propose a novel method named Dynamic Loss Thresholding (DLT). During the training process, DLT records the loss value of each sample and calculates dynamic loss thresholds. Specifically, DLT compares the loss value of each sample with the current loss threshold. Samples with smaller losses can be considered as clean samples with higher probability and vice versa. Then, DLT discards the potentially corrupted labels and further leverages supervised learning techniques. Experiments on CIFAR-10/100 and Clothing1M demonstrate substantial improvements over recent state-of-the-art methods. In addition, we investigate two real-world problems for the first time. Firstly, we propose a novel approach to estimate the noise rates of datasets based on the loss difference between the early and late training stages of DNNs. Secondly, we explore the effect of hard samples (which are difficult to be distinguished) on the process of learning from noisy labels.

Key facts. Cited by 6 papers in the map (data through 13 Oct 2026). Builds on 51 of its 67 references that are in the map. First posted on arXiv 1 Apr 2021.

Builds on

51 of 67 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,321
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,476
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,170
  7. 2013Intriguing properties of neural networkscited by 5,137
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,082
  9. 2016Identity Mappings in Deep Residual Networkscited by 1,962
  10. 2016Understanding deep learning requires rethinking generalizationcited by 1,666
  11. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  12. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
38 more
  1. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,306
  2. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,055
  3. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,055
  4. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  5. 2017A Closer Look at Memorization in Deep Networkscited by 939
  6. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
  7. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  8. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  9. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 509
  10. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
  11. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  12. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
  13. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 415
  14. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
  15. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  16. 2020Meta Pseudo Labelscited by 374
  17. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
  18. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  19. 2019Label Propagation for Deep Semi-supervised Learningcited by 288
  20. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  21. 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
  22. 2015Semi-Supervised Learning with Ladder Networkscited by 247
  23. 2019Billion-scale semi-supervised learning for image classificationcited by 232
  24. 2017Decoupling "when to update" from "how to update"cited by 230
  25. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  26. 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
  27. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  28. 2017CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noisecited by 181
  29. 2017Learning from Noisy Labels with Distillationcited by 160
  30. 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
  31. 2017Learning From Noisy Large-Scale Datasets With Minimal Supervisioncited by 115
  32. 2015Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networkscited by 95
  33. 2018Masking: A New Perspective of Noisy Supervisioncited by 93
  34. 2020A Second-Order Approach to Learning with Instance-Dependent Label Noisecited by 90
  35. 2017Toward Robustness against Label Noise in Training Deep Discriminative Neural Networkscited by 78
  36. 2017Deep Learning from Noisy Image Labels with Quality Embeddingcited by 46
  37. 2017Adversarial Dropout for Supervised and Semi-supervised Learningcited by 38
  38. 2020Weakly Supervised Learning with Side Information for Noisy Labeled Imagescited by 30

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 0; 2026: 3.

6 papers.

  1. 2023Weakly Supervised AUC Optimization: A Unified Partial AUC Approachcited by 11
  2. 2024Hybrid-TTA: Continual Test-time Adaptation via Dynamic Domain Shift Detectioncited by 5
  3. 2022A Robust Optimization Method for Label Noisy Datasets Based on Adaptive Threshold: Adaptive-kcited by 0
  4. 2026How Much Noise Can BERT Handle? Insights from Multilingual Sentence Difficulty Detectioncited by 0
  5. 2026Feature-Label Modal Alignment for Robust Partial Multi-Label Learningcited by 0
  6. 2026On Revisiting Entropy for Identifying Mislabeled Imagescited by 0