mlsanityData to 14 Oct 2026

Early Stopping without a Validation Set

arXiv 1703.09580 ↗First posted 28 Mar 2017cs.LG stat.MLcited by 39

Not on the map. Why?

Abstract (arXiv)

Early stopping is a widely used technique to prevent poor generalization performance when training an over-expressive model by means of gradient-based optimization. To find a good point to halt the optimizer, a common practice is to split the dataset into a training and a smaller validation set to obtain an ongoing estimate of the generalization performance. We propose a novel early stopping criterion based on fast-to-compute local statistics of the computed gradients and entirely removes the need for a held-out validation set. Our experiments show that this is a viable approach in the setting of least-squares and logistic regression, as well as neural networks.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 5; 2023: 7; 2024: 8; 2025: 9; 2026: 5.

39 papers.

Sort:

By area:

  1. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  2. 2025Early Stopping Against Label Noise Without Validation Datacited by 21
  3. 2022It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teachercited by 20
  4. 2025Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stagescited by 17
  5. 2022VICE: Variational Interpretable Concept Embeddingscited by 15
  6. 2023Image-free Classifier Injection for Zero-Shot Classificationcited by 15
  7. 2026Teaching Models to Teach Themselves: Reasoning at the Edge of Learnabilitycited by 13
  8. 2021Channel-Wise Early Stopping without a Validation Set via NNK Polytope Interpolationcited by 11
  9. 2023Beyond Myopia: Learning from Positive and Unlabeled Data through Holistic Predictive Trendscited by 10
  10. 2025Towards Understanding How Knowledge Evolves in Large Vision-Language Modelscited by 10
  11. 2021Disparity Between Batches as a Signal for Early Stoppingcited by 9
  12. 2023How Many Validation Labels Do You Need? Exploring the Design Space of Label-Efficient Model Rankingcited by 8
27 more
  1. 2021Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networkscited by 7
  2. 2024A Layer Selection Approach to Test Time Adaptationcited by 7
  3. 2024A Realistic Protocol for Evaluation of Weakly Supervised Object Localizationcited by 5
  4. 2021Isotonic Data Augmentation for Knowledge Distillationcited by 4
  5. 2022Intersection of Parallels as an Early Stopping Criterioncited by 3
  6. 2024Exploring Selective Layer Fine-Tuning in Federated Learningcited by 3
  7. 2022Improving Meta-Learning Generalization with Activation-Based Early-Stoppingcited by 2
  8. 2023SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruningcited by 2
  9. 2023Addressing caveats of neural persistence with deep graph persistencecited by 1
  10. 2024Differentiating Policies for Non-Myopic Bayesian Optimizationcited by 1
  11. 2024Noisy Early Stopping for Noisy Labelscited by 1
  12. 2025Assessing the Effectiveness of Deep Embeddings for Tree Species Classification in the Dutch Forest Inventorycited by 1
  13. 2025Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Taskscited by 1
  14. 2021Exploiting All Samples in Low-Resource Sentence Classification: Early Stopping and Initialization Parameterscited by 0
  15. 2023Double Descent of Discrepancy: A Task-, Data-, and Model-Agnostic Phenomenoncited by 0
  16. 2023SANE: The phases of gradient descent through Sharpness Adjusted Number of Effective parameterscited by 0
  17. 2024When do Convolutional Neural Networks Stop Learning?cited by 0
  18. 2024Twin: Tuning Learning Rate and Weight Decay of Deep Homogeneous Classifiers without Validationcited by 0
  19. 2024Accelerating Recommender Model Training by Dynamically Skipping Stale Embeddingscited by 0
  20. 2025Is Supervised Learning Really That Different from Unsupervised?cited by 0
  21. 2025GRADSTOP: Early Stopping of Gradient Descent via Posterior Samplingcited by 0
  22. 2025GradES: Significantly Faster Training in Transformers with Gradient-Based Early Stoppingcited by 0
  23. 2025Neural Coherence : Find higher performance to out-of-distribution tasks from few samplescited by 0
  24. 2026Model-Centric Diagnostics: A Framework for Internal State Readoutscited by 0
  25. 2026Information Hidden in Gradients of Regression with Target Noisecited by 0
  26. 2026ScoreStop: Gradient-based early stopping using functional score testscited by 0
  27. 2026SelfOp: An Optimization Algorithm for Self-Improving Security Agentscited by 0