# Iterative Regularization for Learning with Convex Loss Functions

arXiv: https://arxiv.org/abs/1503.08985 · HTML: https://mlsanity.com/paper/1503.08985/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2015.

## Built on by (most cited first, 9 of 9)

- [Optimal Linear Decay Learning Rate Schedules and Further Refinements](https://mlsanity.com/paper/2310.07831/) (2023)
- [Towards Weaker Variance Assumptions for Stochastic Optimization](https://mlsanity.com/paper/2504.09951/) (2025)
- [Learning with Gradient Descent and Weakly Convex Losses](https://mlsanity.com/paper/2101.04968/) (2021)
- [Exponential Tail Local Rademacher Complexity Risk Bounds Without the Bernstein Condition](https://mlsanity.com/paper/2202.11461/) (2022)
- [Random Smoothing Regularization in Kernel Gradient Descent Learning](https://mlsanity.com/paper/2305.03531/) (2023)
- [From inexact optimization to learning via gradient concentration](https://mlsanity.com/paper/2106.05397/) (2021)
- [Iterative regularization in classification via hinge loss diagonal descent](https://mlsanity.com/paper/2212.12675/) (2022)
- [Lp- and Risk Consistency of Localized SVMs](https://mlsanity.com/paper/2305.09385/) (2023)
- [Self-Regularized Learning Methods](https://mlsanity.com/paper/2603.17160/) (2026)

## Abstract (arXiv)

We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stopping an empirical iteration. We consider a nonparametric setting, in the framework of reproducing kernel Hilbert spaces, and prove finite sample bounds on the excess risk under general regularity conditions. Our study provides a new class of efficient regularized learning algorithms and gives insights on the interplay between statistics and optimization in machine learning.