# Adaptive proximal gradient methods are universal without approximation

arXiv: https://arxiv.org/abs/2402.06271 · HTML: https://mlsanity.com/paper/2402.06271/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 9 Feb 2024.

## Builds on (most cited first, 22 of 22)

- [On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes](https://mlsanity.com/paper/1805.08114/) (2018)
- [Adaptive Gradient Descent without Descent](https://mlsanity.com/paper/1910.09529/) (2019)
- [Adaptive Proximal Gradient Method for Convex Optimization](https://mlsanity.com/paper/2308.02261/) (2023)
- [Super-Universal Regularized Newton Method](https://mlsanity.com/paper/2208.05888/) (2022)
- [Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradient](https://mlsanity.com/paper/2301.04431/) (2023)
- [A simple uniformly optimal method without line search for convex optimization](https://mlsanity.com/paper/2310.10082/) (2023)
- [RSG: Beating Subgradient Method without Smoothness and Strong Convexity](https://mlsanity.com/paper/1512.03107/) (2015)
- [Primal-dual accelerated gradient methods with small-dimensional relaxation oracle](https://mlsanity.com/paper/1809.05895/) (2018)
- [Unregularized Online Learning Algorithms with General Loss Functions](https://mlsanity.com/paper/1503.00623/) (2015)
- [Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Method](https://mlsanity.com/paper/2206.06900/) (2022)
- [On Optimal Universal First-Order Methods for Minimizing Heterogeneous Sums](https://mlsanity.com/paper/2208.08549/) (2022)
- [Inexact Model: A Framework for Optimization and Variational Inequalities](https://mlsanity.com/paper/1902.00990/) (2019)
- [AdaBB: Adaptive Barzilai-Borwein Method for Convex Optimization](https://mlsanity.com/paper/2401.08024/) (2024)
- [Three Operator Splitting with Subgradients, Stochastic Gradients, and Adaptive Learning Rates](https://mlsanity.com/paper/2110.03274/) (2021)
- [A Universal Primal-Dual Convex Optimization Framework](https://mlsanity.com/paper/1502.03123/) (2015)
- [Anisotropic Proximal Gradient](https://mlsanity.com/paper/2210.15531/) (2022)
- [Generalized Uniformly Optimal Methods for Nonlinear Programming](https://mlsanity.com/paper/1508.07384/) (2015)
- [Global Convergence Analysis of the Power Proximal Point and Augmented Lagrangian Method](https://mlsanity.com/paper/2312.12205/) (2023)
- [Universal Intermediate Gradient Method for Convex Problems with Inexact Oracle](https://mlsanity.com/paper/1712.06036/) (2017)
- [Nonlocal p-Laplacian Variational problems on graphs](https://mlsanity.com/paper/1810.12817/) (2018)
- [On the convergence of adaptive first order methods: proximal gradient and alternating minimization algorithms](https://mlsanity.com/paper/2311.18431/) (2023)
- [Anisotropic Proximal Point Algorithm](https://mlsanity.com/paper/2312.09834/) (2023)

## Built on by (most cited first, 4 of 4)

- [Proximal Interacting Particle Langevin Algorithms](https://mlsanity.com/paper/2406.14292/) (2024)
- [Learning Equilibria in Adversarial Team Markov Games: A Nonconvex-Hidden-Concave Min-Max Optimization Problem](https://mlsanity.com/paper/2410.05673/) (2024)
- [The inexact power augmented Lagrangian method for constrained nonconvex optimization](https://mlsanity.com/paper/2410.20153/) (2024)
- [Safeguarding adaptive methods: global convergence of Barzilai-Borwein and other stepsize choices](https://mlsanity.com/paper/2404.09617/) (2024)

## Abstract (arXiv)

We show that adaptive proximal gradient methods for convex problems are not restricted to traditional Lipschitzian assumptions. Our analysis reveals that a class of linesearch-free methods is still convergent under mere local H"older gradient continuity, covering in particular continuously differentiable semi-algebraic functions. To mitigate the lack of local Lipschitz continuity, popular approaches revolve around varepsilon-oracles and/or linesearch procedures. In contrast, we exploit plain H"older inequalities not entailing any approximation, all while retaining the linesearch-free nature of adaptive schemes. Furthermore, we prove full sequence convergence without prior knowledge of local H"older constants nor of the order of H"older continuity. Numerical experiments make comparisons with baseline methods on diverse tasks from machine learning covering both the locally and the globally H"older setting.