# Optimized first-order methods for smooth convex minimization

arXiv: https://arxiv.org/abs/1406.5468 · HTML: https://mlsanity.com/paper/1406.5468/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2014.

## Built on by (most cited first, 18 of 18)

- [Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities and Connections With Cocoercivity](https://mlsanity.com/paper/2110.04261/) (2021)
- [Provably Faster Gradient Descent via Long Steps](https://mlsanity.com/paper/2307.06324/) (2023)
- [PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming](https://mlsanity.com/paper/2406.01908/) (2024)
- [A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip](https://mlsanity.com/paper/2106.07644/) (2021)
- [PEPit: computer-assisted worst-case analyses of first-order optimization methods in Python](https://mlsanity.com/paper/2201.04040/) (2022)
- [Accelerating Value Iteration with Anchoring](https://mlsanity.com/paper/2305.16569/) (2023)
- [Randomized Block-Coordinate Optimistic Gradient Algorithms for Root-Finding Problems](https://mlsanity.com/paper/2301.03113/) (2023)
- [Practical Schemes for Finding Near-Stationary Points of Convex Finite-Sums](https://mlsanity.com/paper/2105.12062/) (2021)
- [A lower bound for stepsize-based acceleration of gradient descent](https://mlsanity.com/paper/2608.10418/) (2026)
- [Memory-Constrained Algorithms for Convex Optimization via Recursive Cutting-Planes](https://mlsanity.com/paper/2306.10096/) (2023)
- [Restarts subject to approximate sharpness: A parameter-free and optimal scheme for first-order methods](https://mlsanity.com/paper/2301.02268/) (2023)
- [Incorporating the Barzilai-Borwein Adaptive Step Size into Sugradient Methods for Deep Network Training](https://mlsanity.com/paper/2205.13711/) (2022)
- [Poisson-Gaussian Holographic Phase Retrieval with Score-based Image Prior](https://mlsanity.com/paper/2305.07712/) (2023)
- [Error estimates between SGD with momentum and underdamped Langevin diffusion](https://mlsanity.com/paper/2410.17297/) (2024)
- [An Improved Optimal Proximal Gradient Algorithm for Non-Blind Image Deblurring](https://mlsanity.com/paper/2502.07602/) (2025)
- [A Triple-Inertial Accelerated Alternating Optimization Method for Deep Learning Training](https://mlsanity.com/paper/2503.08489/) (2025)
- [Dual Optimization for Kolmogorov Model Learning Using Enhanced Gradient Descent](https://mlsanity.com/paper/2107.05011/) (2021)
- [Neural Network Training via Stochastic Alternating Minimization with Trainable Step Sizes](https://mlsanity.com/paper/2508.04193/) (2025)

## Abstract (arXiv)

We introduce new optimized first-order methods for smooth unconstrained convex minimization. Drori and Teboulle recently described a numerical method for computing the N-iteration optimal step coefficients in a class of first-order algorithms that includes gradient methods, heavy-ball methods, and Nesterov's fast gradient methods. However, Drori and Teboulle's numerical method is computationally expensive for large N, and the corresponding numerically optimized first-order algorithm requires impractical memory and computation for large-scale optimization problems. In this paper, we propose optimized first-order algorithms that achieve a convergence bound that is two times smaller than for Nesterov's fast gradient methods; our bound is found analytically and refines the numerical bound. Furthermore, the proposed optimized first-order methods have efficient recursive forms that are remarkably similar to Nesterov's fast gradient methods.