mlsanityData to 7 Oct 2026

Early stopping for kernel boosting algorithms: A general analysis with localized complexities

arXiv 1707.01543 ↗First posted 5 Jul 2017stat.ML cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Early stopping of iterative algorithms is a widely-used form of regularization in statistics, commonly used in conjunction with boosting and related gradient-type algorithms. Although consistency results have been established in some settings, such estimators are less well-understood than their analogues based on penalized regularization. In this paper, for a relatively broad class of loss functions and boosting algorithms (including L2-boost, LogitBoost and AdaBoost, among others), we exhibit a direct connection between the performance of a stopped iterate and the localized Gaussian complexity of the associated function class. This connection allows us to show that local fixed point analysis of Gaussian or Rademacher complexities, now standard in the analysis of penalized estimators, can be used to derive optimal stopping rules. We derive such stopping rules in detail for various kernel classes, and illustrate the correspondence of our theory with practice for Sobolev kernel classes.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 3; 2024: 5; 2025: 4; 2026: 8.

27 papers.

Sort:

By area:

  1. 2024Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalizationcited by 39
  2. 2023Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimensioncited by 26
  3. 2021Optimal policy evaluation using kernel-based temporal difference methodscited by 24
  4. 2022Off-Policy Evaluation for Episodic Partially Observable Markov Decision Processes under Non-Parametric Modelscited by 18
  5. 2022Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descentcited by 11
  6. 2022Exponential Tail Local Rademacher Complexity Risk Bounds Without the Bernstein Conditioncited by 9
  7. 2023Strong inductive biases provably prevent harmless interpolationcited by 9
  8. 2022Risk Bounds of Multi-Pass SGD for Least Squares in the Interpolation Regimecited by 8
  9. 2023Random Smoothing Regularization in Kernel Gradient Descent Learningcited by 6
  10. 2021From inexact optimization to learning via gradient concentrationcited by 4
  11. 2024Failures and Successes of Cross-Validation for Early-Stopped Gradient Descentcited by 4
  12. 2024Reliable and scalable variable importance estimation via warm-start and early stoppingcited by 2
15 more
  1. 2025Basic Inequalities for First-Order Optimization with Applications to Statistical Risk Analysiscited by 2
  2. 2022Iterative regularization for low complexity regularizerscited by 1
  3. 2026A Theoretical Framework for LLM Fine-tuning Using Early Stopping for Non-random Initializationcited by 1
  4. 2026Beyond Cross-Validation: Adaptive Parameter Selection for Kernel-Based Gradient Descentscited by 1
  5. 2026Residual-as-Teacher: Mitigating Bias Propagation in Student--Teacher Estimationcited by 1
  6. 2024Early stopping by correlating online indicators in neural networkscited by 0
  7. 2024Absolute convergence and error thresholds in non-active adaptive samplingcited by 0
  8. 2025Uniform convergence of the smooth calibration error and its relationship with functional gradientcited by 0
  9. 2025A Ridge Too Far: Correcting Over-Shrinkage via Negative Regularizationcited by 0
  10. 2025GRADSTOP: Early Stopping of Gradient Descent via Posterior Samplingcited by 0
  11. 2026Don't stop me now: How Validation Criteria Affect Checkpoint Selection and Early Stoppingcited by 0
  12. 2026ScoreStop: Gradient-based early stopping using functional score testscited by 0
  13. 2026The Geometry of Saturation: Effective Rank Predicts When Labels Stop Helping in Few-Shot Classificationcited by 0
  14. 2026Adaptive Regularization for Random Features: A Neighboring Early-Stopping Rule with Oracle-Rate Guaranteescited by 0
  15. 2026A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networkscited by 0