# Instance-dependent Early Stopping

arXiv: https://arxiv.org/abs/2502.07547 · HTML: https://mlsanity.com/paper/2502.07547/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2025.

## Built on by (most cited first, 10 of 10)

- [Toward Robust Non-Transferable Learning: A Survey and Benchmark](https://mlsanity.com/paper/2502.13593/) (2025)
- [Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examples](https://mlsanity.com/paper/2502.08227/) (2025)
- [Progressive Data Dropout: An Embarrassingly Simple Approach to Faster Training](https://mlsanity.com/paper/2505.22342/) (2025)
- [mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT](https://mlsanity.com/paper/2603.21606/) (2026)
- [Mitigating Mismatch within Reference-based Preference Optimization](https://mlsanity.com/paper/2602.11902/) (2026)
- [Adaptive Data Dropout: Towards Self-Regulated Learning in Deep Neural Networks](https://mlsanity.com/paper/2604.12945/) (2026)
- [Reducing Class Bias In Data-Balanced Datasets Through Hardness-Based Resampling](https://mlsanity.com/paper/2504.07031/) (2025)
- [GradES: Significantly Faster Training in Transformers with Gradient-Based Early Stopping](https://mlsanity.com/paper/2509.01842/) (2025)
- [Beyond Loss Values: Robust Dynamic Pruning via Loss Trajectory Alignment](https://mlsanity.com/paper/2604.07306/) (2026)
- [Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling](https://mlsanity.com/paper/2605.23198/) (2026)

## Abstract (arXiv)

In machine learning practice, early stopping has been widely used to regularize models and can save computational costs by halting the training process when the model's performance on a validation set stops improving. However, conventional early stopping applies the same stopping criterion to all instances without considering their individual learning statuses, which leads to redundant computations on instances that are already well-learned. To further improve the efficiency, we propose an Instance-dependent Early Stopping (IES) method that adapts the early stopping mechanism from the entire training set to the instance level, based on the core principle that once the model has mastered an instance, the training on it should stop. IES considers an instance as mastered if the second-order differences of its loss value remain within a small range around zero. This offers a more consistent measure of an instance's learning status compared with directly using the loss value, and thus allows for a unified threshold to determine when an instance can be excluded from further backpropagation. We show that excluding mastered instances from backpropagation can increase the gradient norms, thereby accelerating the decrease of the training loss and speeding up the training process. Extensive experiments on benchmarks demonstrate that IES method can reduce backpropagation instances by 10%-50% while maintaining or even slightly improving the test accuracy and transfer learning performance of a model.