# Optimizing ADMM and Over-Relaxed ADMM Parameters for Linear Quadratic Problems

arXiv: https://arxiv.org/abs/2401.00657 · HTML: https://mlsanity.com/paper/2401.00657/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 1 Jan 2024.

## Builds on (most cited first, 9 of 9)

- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [OSQP: An Operator Splitting Solver for Quadratic Programs](https://mlsanity.com/paper/1711.08013/) (2017)
- [Learning a Model-Driven Variational Network for Deformable Image Registration](https://mlsanity.com/paper/2105.12227/) (2021)
- [ADMM for Efficient Deep Learning with Global Convergence](https://mlsanity.com/paper/1905.13611/) (2019)
- [Optimal parameter selection for the alternating direction method of multipliers (ADMM): quadratic problems](https://mlsanity.com/paper/1306.2454/) (2013)
- [Nesterov Accelerated ADMM for Fast Diffeomorphic Image Registration](https://mlsanity.com/paper/2109.12688/) (2021)
- [Adaptive ADMM with Spectral Penalty Parameter Selection](https://mlsanity.com/paper/1605.07246/) (2016)
- [ADMM Penalty Parameter Selection by Residual Balancing](https://mlsanity.com/paper/1704.06209/) (2017)
- [An Explicit Rate Bound for the Over-Relaxed ADMM](https://mlsanity.com/paper/1512.02063/) (2015)

## Abstract (arXiv)

The Alternating Direction Method of Multipliers (ADMM) has gained significant attention across a broad spectrum of machine learning applications. Incorporating the over-relaxation technique shows potential for enhancing the convergence rate of ADMM. However, determining optimal algorithmic parameters, including both the associated penalty and relaxation parameters, often relies on empirical approaches tailored to specific problem domains and contextual scenarios. Incorrect parameter selection can significantly hinder ADMM's convergence rate. To address this challenge, in this paper we first propose a general approach to optimize the value of penalty parameter, followed by a novel closed-form formula to compute the optimal relaxation parameter in the context of linear quadratic problems (LQPs). We then experimentally validate our parameter selection methods through random instantiations and diverse imaging applications, encompassing diffeomorphic image registration, image deblurring, and MRI reconstruction.