On the convergence of adaptive first order methods: proximal gradient and alternating minimization algorithms
Not on the map. Why?
Abstract (arXiv)
Building upon recent works on linesearch-free adaptive proximal gradient methods, this paper proposes adaPGq,r, a framework that unifies and extends existing results by providing larger stepsize policies and improved lower bounds. Different choices of the parameters q and r are discussed and the efficacy of the resulting methods is demonstrated through numerical simulations. In an attempt to better understand the underlying theory, its convergence is established in a more general setting that allows for time-varying parameters. Finally, an adaptive alternating minimization algorithm is presented by exploring the dual setting. This algorithm not only incorporates additional adaptivity, but also expands its applicability beyond standard strongly convex settings.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 16 of its 27 references that are in the map. First posted on arXiv 30 Nov 2023.
Builds on
16 of 27 references.
References · 16
- 2018AdaGrad stepsizes: Sharp convergence over nonconvex landscapescited by 182
- 2018On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizescited by 156
- 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
- 2019Adaptive Gradient Descent without Descentcited by 79
- 2018Golden Ratio Algorithms for Variational Inequalitiescited by 38
- 2023Adaptive Proximal Gradient Method for Convex Optimizationcited by 21
- 2023Acceleration by Stepsize Hedging II: Silver Stepsize Schedule for Smooth Convex Optimizationcited by 20
- 2023Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradientcited by 19
- 2020Adaptive Gradient Methods for Constrained Convex Optimization and Variational Inequalitiescited by 18
- 2023A simple uniformly optimal method without line search for convex optimizationcited by 18
- 2022Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Methodcited by 9
- 2023Accelerated Gradient Descent via Long Stepscited by 9
4 more
- 2021Three Operator Splitting with Subgradients, Stochastic Gradients, and Adaptive Learning Ratescited by 6
- 2021Proximal Gradient Algorithms under Local Lipschitz Gradient Continuity: A Convergence and Robustness Analysis of PANOCcited by 4
- 2016The variable metric forward-backward splitting algorithm under mild differentiability assumptionscited by 3
- 2023On the convergence of proximal gradient methods for convex simple bilevel optimizationcited by 2
Built on by
Citing papers per year: 2024: 2.
2 papers.