# Learning Algorithm Hyperparameters for Fast Parametric Convex Optimization

arXiv: https://arxiv.org/abs/2411.15717 · HTML: https://mlsanity.com/paper/2411.15717/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 6 references that are in the map.
- First posted on arXiv 24 Nov 2024.

## Built on by (most cited first, 7 of 7)

- [Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfolding](https://mlsanity.com/paper/2512.01565/) (2025)
- [Learning to accelerate distributed ADMM using graph neural networks](https://mlsanity.com/paper/2509.05288/) (2025)
- [Warm-starting active-set solvers using graph neural networks](https://mlsanity.com/paper/2511.13174/) (2025)
- [Learning Over-Relaxation Policies for ADMM with Convergence Guarantees](https://mlsanity.com/paper/2604.26932/) (2026)
- [Deep-Unfolded Coordination](https://mlsanity.com/paper/2606.19920/) (2026)
- [Reinforcement Learning to Accelerate Primal-Dual Hybrid Gradient for Linear Programming](https://mlsanity.com/paper/2610.01546/) (2026)
- [A Scalable Learned Iterative Solver for Nonlinear MPC via Temporal Decomposition](https://mlsanity.com/paper/2610.13480/) (2026)

## Abstract (arXiv)

We introduce a machine-learning framework to learn the hyperparameter sequence of first-order methods (e.g., the step sizes in gradient descent) to quickly solve parametric convex optimization problems. Our computational architecture amounts to running fixed-point iterations where the hyperparameters are the same across all parametric instances and consists of two phases. In the first step-varying phase the hyperparameters vary across iterations, while in the second steady-state phase the hyperparameters are constant across iterations. Our learned optimizer is flexible in that it can be evaluated on any number of iterations and is guaranteed to converge to an optimal solution. To train, we minimize the mean square error to a ground truth solution. In the case of gradient descent, the one-step optimal step size is the solution to a least squares problem, and in the case of unconstrained quadratic minimization, we can compute the two and three-step optimal solutions in closed-form. In other cases, we backpropagate through the algorithm steps to minimize the training objective after a given number of steps. We show how to learn hyperparameters for several popular algorithms: gradient descent, proximal gradient descent, and two ADMM-based solvers: OSQP and SCS. We use a sample convergence bound to obtain generalization guarantees for the performance of our learned algorithm for unseen data, providing both lower and upper bounds. We showcase the effectiveness of our method with many examples, including ones from control, signal processing, and machine learning. Remarkably, our approach is highly data-efficient in that we only use 10 problem instances to train the hyperparameters in all of our examples.