mlsanityData to 14 Oct 2026

Late Stopping: Avoiding Confidently Learning from Mislabeled Examples

arXiv 2308.13862 ↗First posted 26 Aug 2023cs.LG cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Sample selection is a prevalent method in learning with noisy labels, where small-loss data are typically considered as correctly labeled data. However, this method may not effectively identify clean hard examples with large losses, which are critical for achieving the model's close-to-optimal generalization performance. In this paper, we propose a new framework, Late Stopping, which leverages the intrinsic robust learning ability of DNNs through a prolonged training process. Specifically, Late Stopping gradually shrinks the noisy dataset by removing high-probability mislabeled examples while retaining the majority of clean hard examples in the training set throughout the learning process. We empirically observe that mislabeled and clean examples exhibit differences in the number of epochs required for them to be consistently and correctly classified, and thus high-probability mislabeled examples can be removed. Experimental results on benchmark-simulated and real-world noisy datasets demonstrate that the proposed method outperforms state-of-the-art counterparts.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 40 of its 58 references that are in the map. First posted on arXiv 26 Aug 2023.

Builds on

40 of 58 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  3. 2017A Closer Look at Memorization in Deep Networkscited by 942
  4. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
  5. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  6. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  7. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  8. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  9. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  10. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  11. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 416
  12. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
28 more
  1. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
  2. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
  3. 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 261
  4. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 256
  5. 2021A Survey on Deep Semi-supervised Learningcited by 251
  6. 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
  7. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  8. 2017Deep Learning is Robust to Massive Label Noisecited by 165
  9. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  10. 2019Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networkscited by 159
  11. 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
  12. 2021Deep Learning Through the Lens of Example Difficultycited by 141
  13. 2020A Survey of Label-noise Representation Learning: Past, Present and Futurecited by 122
  14. 2015Learning with Symmetric Label Noise: The Importance of Being Unhingedcited by 120
  15. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 113
  16. 2018Iterative Learning with Open-set Noisy Labelscited by 108
  17. 2020Characterizing Structural Regularities of Labeled Data in Overparameterized Modelscited by 93
  18. 2019Curriculum Loss: Robust Learning and Generalization against Label Corruptioncited by 88
  19. 2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
  20. 2022UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learningcited by 77
  21. 2020A Topological Filter for Learning with Label Noisecited by 76
  22. 2021NGC: A Unified Framework for Learning with Open-World Noisy Datacited by 58
  23. 2020Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularizationcited by 49
  24. 2019Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applicationscited by 43
  25. 2022Harnessing Out-Of-Distribution Examples via Augmenting Content and Stylecited by 43
  26. 2022Characterizing Datapoints via Second-Split Forgettingcited by 37
  27. 2022Self-Filtering: A Noise-Aware Sample Selection for Label Noise with Confidence Penalizationcited by 33
  28. 2017Learning with Noise: Enhance Distantly Supervised Relation Extraction with Dynamic Transition Matrixcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 8; 2026: 3.

17 papers.

Sort:

By area:

  1. 2025Understanding and Enhancing the Transferability of Jailbreaking Attackscited by 22
  2. 2025Early Stopping Against Label Noise Without Validation Datacited by 21
  3. 2023On the Over-Memorization During Natural, Robust and Catastrophic Overfittingcited by 20
  4. 2023FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learningcited by 20
  5. 2025FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEctioncited by 11
  6. 2024Mitigating Label Noise on Graph via Topological Sample Selectioncited by 6
  7. 2025Handling Label Noise via Instance-Level Difficulty Modeling and Dynamic Optimizationcited by 6
  8. 2025Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examplescited by 4
  9. 2025ChronoSelect: Robust Learning with Noisy Labels via Dynamics Temporal Memorycited by 4
  10. 2024Understanding and Mitigating the Bias in Sample Selection for Learning with Noisy Labelscited by 1
  11. 2024Mislabeled examples detection viewed as probing machine learning models: concepts, survey and extensive benchmarkcited by 1
  12. 2025Benchmarking noisy label detection methodscited by 1
5 more
  1. 2025Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Taskscited by 1
  2. 2026Mitigating Mismatch within Reference-based Preference Optimizationcited by 1
  3. 2026The Blind Spot in 2D Infants' Pose Estimation:Robust Learning from Noisy Annotationscited by 1
  4. 2024Jump-teaching: Combating Sample Selection Bias via Temporal Disagreementcited by 0
  5. 2026Variational Rectification Inference for Learning with Noisy Labelscited by 0