mlsanityData to 7 Oct 2026

Early Stopping Against Label Noise Without Validation Data

arXiv 2502.07551 ↗First posted 11 Feb 2025cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Early stopping methods in deep learning face the challenge of balancing the volume of training and validation data, especially in the presence of label noise. Concretely, sparing more data for validation from training data would limit the performance of the learned model, yet insufficient validation data could result in a sub-optimal selection of the desired model. In this paper, we propose a novel early stopping method called Label Wave, which does not require validation data for selecting the desired model in the presence of label noise. It works by tracking the changes in the model's predictions on the training set during the training process, aiming to halt training before the model unduly fits mislabeled data. This method is empirically supported by our observation that minimum fluctuations in predictions typically occur at the training epoch before the model excessively fits mislabeled data. Through extensive experiments, we show both the effectiveness of the Label Wave method across various settings and its capability to enhance the performance of existing methods for learning with noisy labels.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 55 of its 81 references that are in the map. First posted on arXiv 11 Feb 2025.

Builds on

55 of 81 references.

References · 55

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  8. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  9. 2017A Closer Look at Memorization in Deep Networkscited by 936
  10. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  11. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  12. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
38 more
  1. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  2. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
  3. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  4. 2019Benign Overfitting in Linear Regressioncited by 544
  5. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  6. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  7. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
  8. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  9. 2014On the Number of Linear Regions of Deep Neural Networkscited by 378
  10. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  11. 2018Stronger generalization bounds for deep nets via a compression approachcited by 306
  12. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  13. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
  14. 2017High-dimensional dynamics of generalization error in neural networkscited by 238
  15. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 229
  16. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 217
  17. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  18. 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
  19. 2020A Survey of Label-noise Representation Learning: Past, Present and Futurecited by 121
  20. 2015Learning with Symmetric Label Noise: The Importance of Being Unhingedcited by 120
  21. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 112
  22. 2018Predicting the Generalization Gap in Deep Networks with Margin Distributionscited by 111
  23. 2020Are Labels Always Necessary for Classifier Accuracy Evaluation?cited by 104
  24. 2018On the importance of single directions for generalizationcited by 99
  25. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  26. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  27. 2020Computing the Testing Error without a Testing Setcited by 54
  28. 2019How does Early Stopping Help Generalization against Label Noise?cited by 41
  29. 2017Early Stopping without a Validation Setcited by 39
  30. 2022Characterizing Datapoints via Second-Split Forgettingcited by 37
  31. 2022Self-Filtering: A Noise-Aware Sample Selection for Label Noise with Confidence Penalizationcited by 33
  32. 2023Machine Vision Therapy: Multimodal Large Language Models Can Enhance Visual Robustness via Denoising In-Context Learningcited by 27
  33. 2020Speedy Performance Estimation for Neural Architecture Searchcited by 24
  34. 2022SELC: Self-Ensemble Label Correction Improves Learning with Noisy Labelscited by 24
  35. 2015Early Stopping is Nonparametric Variational Inferencecited by 23
  36. 2020Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labelscited by 20
  37. 2021RATT: Leveraging Unlabeled Data to Guarantee Generalizationcited by 20
  38. 2020A Bayesian Perspective on Training Speed and Model Selectioncited by 19

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 7.

21 papers.

Sort:

By area:

  1. 2025FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEctioncited by 11
  2. 2025Knowledge Integration for Physics-informed Symbolic Regression Using Pre-trained Large Language Modelscited by 7
  3. 2026CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbackscited by 6
  4. 2025Handling Label Noise via Instance-Level Difficulty Modeling and Dynamic Optimizationcited by 5
  5. 2025Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examplescited by 4
  6. 2026Optimal Transport for LLM Reward Modeling from Noisy Preferencecited by 3
  7. 2025Robust Classification with Noisy Labels Based on Posterior Maximizationcited by 2
  8. 2025LoD: Loss-difference OOD Detection by Intentionally Label-Noisifying Unlabeled Wild Datacited by 1
  9. 2025Early-stopping for Transformer model trainingcited by 1
  10. 2025Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Taskscited by 1
  11. 2026Mitigating Mismatch within Reference-based Preference Optimizationcited by 1
  12. 2025crowd-hpo: Realistic Hyperparameter Optimization and Benchmarking for Learning from Crowds with Noisy Labelscited by 0
9 more
  1. 2025Is Supervised Learning Really That Different from Unsupervised?cited by 0
  2. 2025Neural Velocity for hyperparameter tuningcited by 0
  3. 2025CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labelscited by 0
  4. 2025GRADSTOP: Early Stopping of Gradient Descent via Posterior Samplingcited by 0
  5. 2025GradES: Significantly Faster Training in Transformers with Gradient-Based Early Stoppingcited by 0
  6. 2026MeGU: Machine-Guided Unlearning with Target Feature Disentanglementcited by 0
  7. 2026On Revisiting Entropy for Identifying Mislabeled Imagescited by 0
  8. 2026The Blind Spot in 2D Infants' Pose Estimation:Robust Learning from Noisy Annotationscited by 0
  9. 2026A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networkscited by 0