# How does Early Stopping Help Generalization against Label Noise?

arXiv: https://arxiv.org/abs/1911.08059 · HTML: https://mlsanity.com/paper/1911.08059/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2019.

## Built on by (most cited first, 30 of 41)

- [Deep Learning with Label Differential Privacy](https://mlsanity.com/paper/2102.06062/) (2021)
- [Robust Training under Label Noise by Over-parameterization](https://mlsanity.com/paper/2202.14026/) (2022)
- [FINE Samples for Learning with Noisy Labels](https://mlsanity.com/paper/2102.11628/) (2021)
- [Understanding Instance-Level Label Noise: Disparate Impacts and Treatments](https://mlsanity.com/paper/2102.05336/) (2021)
- [Early Stopping Against Label Noise Without Validation Data](https://mlsanity.com/paper/2502.07551/) (2025)
- [Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection](https://mlsanity.com/paper/2111.14932/) (2021)
- [Instance-Dependent Noisy Label Learning via Graphical Modelling](https://mlsanity.com/paper/2209.00906/) (2022)
- [Disparity Between Batches as a Signal for Early Stopping](https://mlsanity.com/paper/2107.06665/) (2021)
- [Sample Prior Guided Robust Model Learning to Suppress Noisy Labels](https://mlsanity.com/paper/2112.01197/) (2021)
- [Improving Medical Image Classification in Noisy Labels Using Only Self-supervised Pretraining](https://mlsanity.com/paper/2308.04551/) (2023)
- [Tackling Noisy Labels with Network Parameter Additive Decomposition](https://mlsanity.com/paper/2403.13241/) (2024)
- [Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classification](https://mlsanity.com/paper/2106.11486/) (2021)
- [Label Noise-Robust Learning using a Confidence-Based Sieving Strategy](https://mlsanity.com/paper/2210.05330/) (2022)
- [Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization](https://mlsanity.com/paper/2303.10758/) (2023)
- [Confidence Adaptive Regularization for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2108.08212/) (2021)
- [A Survey of Dataset Refinement for Problems in Computer Vision Datasets](https://mlsanity.com/paper/2210.11717/) (2022)
- [PLReMix: Combating Noisy Labels with Pseudo-Label Relaxed Contrastive Representation Learning](https://mlsanity.com/paper/2402.17589/) (2024)
- [Learning to Combat Noisy Labels via Classification Margins](https://mlsanity.com/paper/2102.00751/) (2021)
- [A Gift from Label Smoothing: Robust Training with Adaptive Label Smoothing via Auxiliary Classifier under Label Noise](https://mlsanity.com/paper/2206.07277/) (2022)
- [Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervision](https://mlsanity.com/paper/2306.07036/) (2023)
- [Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examples](https://mlsanity.com/paper/2502.08227/) (2025)
- [How Does Heterogeneous Label Noise Impact Generalization in Neural Nets?](https://mlsanity.com/paper/2106.15475/) (2021)
- [Is the Information Bottleneck Robust Enough? Towards Label-Noise Resistant Information Bottleneck Learning](https://mlsanity.com/paper/2512.10573/) (2025)
- [A Human-in-the-Loop Label Error Detection Framework Applied to Arabic-Script HTR Datasets](https://mlsanity.com/paper/2601.16713/) (2026)
- [CNTN: Cyclic Noise-tolerant Network for Gait Recognition](https://mlsanity.com/paper/2210.06910/) (2022)
- [Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play](https://mlsanity.com/paper/2302.05807/) (2023)
- [Training on Synthetic Data Beats Real Data in Multimodal Relation Extraction](https://mlsanity.com/paper/2312.03025/) (2023)
- [Learning from Different Samples: A Source-free Framework for Semi-supervised Domain Adaptation](https://mlsanity.com/paper/2411.06665/) (2024)
- [Double Descent Optimization Pattern and Aliasing: Caveats of Noisy Labels](https://mlsanity.com/paper/2106.02100/) (2021)
- [How does self-supervised pretraining improve robustness against noisy labels across various medical image classification datasets?](https://mlsanity.com/paper/2401.07990/) (2024)

## Abstract (arXiv)

Noisy labels are very common in real-world training data, which lead to poor generalization on test data because of overfitting to the noisy labels. In this paper, we claim that such overfitting can be avoided by "early stopping" training a deep neural network before the noisy labels are severely memorized. Then, we resume training the early stopped network using a "maximal safe set," which maintains a collection of almost certainly true-labeled samples at each epoch since the early stop point. Putting them all together, our novel two-phase training method, called Prestopping, realizes noise-free training under any type of label noise for practical use. Extensive experiments using four image benchmark data sets verify that our method significantly outperforms four state-of-the-art methods in test error by 0.4-8.2 percent points under existence of real-world noise.