# EarlyStopping: Implicit Regularization for Iterative Learning Procedures in Python

arXiv: https://arxiv.org/abs/2503.16753 · HTML: https://mlsanity.com/paper/2503.16753/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 38 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Builds on (most cited first, 7 of 7)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Consistency of random forests](https://mlsanity.com/paper/1405.2881/) (2014)
- [Large Scale Prediction with Decision Trees](https://mlsanity.com/paper/2104.13881/) (2021)
- [Analyzing the discrepancy principle for kernelized spectral filter learning algorithms](https://mlsanity.com/paper/2004.08436/) (2020)
- [Model selection for high-dimensional linear regression with dependent observations](https://mlsanity.com/paper/1906.07395/) (2019)
- [Comparing regularisation paths of (conjugate) gradient estimators in ridge regression](https://mlsanity.com/paper/2503.05542/) (2025)

## Built on by (most cited first, 1 of 1)

- [A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networks](https://mlsanity.com/paper/2609.21017/) (2026)

## Abstract (arXiv)

Iterative learning procedures are ubiquitous in machine learning and modern statistics. Regularision is typically required to prevent inflating the expected loss of a procedure in later iterations via the propagation of noise inherent in the data. Significant emphasis has been placed on achieving this regularisation implicitly by stopping procedures early. The EarlyStopping-package provides a toolbox of (in-sample) sequential early stopping rules for several well-known iterative estimation procedures, such as truncated SVD, Landweber (gradient descent), conjugate gradient descent, L2-boosting and regression trees. One of the central features of the package is that the algorithms allow the specification of the true data-generating process and keep track of relevant theoretical quantities. In this paper, we detail the principles governing the implementation of the EarlyStopping-package and provide a survey of recent foundational advances in the theoretical literature. We demonstrate how to use the EarlyStopping-package to explore core features of implicit regularisation and replicate results from the literature.