# Accelerating Multi-Block Constrained Optimization Through Learning to Optimize

arXiv: https://arxiv.org/abs/2409.17320 · HTML: https://mlsanity.com/paper/2409.17320/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 63 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 26 of 26)

- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Optimal transport mapping via input convex neural networks](https://mlsanity.com/paper/1908.10962/) (2019)
- [Plug-and-Play Methods Provably Converge with Properly Trained Denoisers](https://mlsanity.com/paper/1905.05406/) (2019)
- [Stochastic Optimization for Large-scale Optimal Transport](https://mlsanity.com/paper/1605.08527/) (2016)
- [Neural Optimal Transport](https://mlsanity.com/paper/2201.12220/) (2022)
- [One Network to Solve Them All --- Solving Linear Inverse Problems using Deep Projection Models](https://mlsanity.com/paper/1703.09912/) (2017)
- [Learning step sizes for unfolded sparse coding](https://mlsanity.com/paper/1905.11071/) (2019)
- [Model Learning: Primal Dual Networks for Fast MR imaging](https://mlsanity.com/paper/1908.02426/) (2019)
- [Differentiable Linearized ADMM](https://mlsanity.com/paper/1905.06179/) (2019)
- [A highly efficient semismooth Newton augmented Lagrangian method for solving Lasso problems](https://mlsanity.com/paper/1607.05428/) (2016)
- [Understanding the Learned Iterative Soft Thresholding Algorithm with matrix factorization](https://mlsanity.com/paper/1706.01338/) (2017)
- [LSALSA: Accelerated Source Separation via Learned Sparse Coding](https://mlsanity.com/paper/1802.06875/) (2018)
- [On the Equivalence of Inexact Proximal ALM and ADMM for a Class of Convex Composite Programming](https://mlsanity.com/paper/1803.10803/) (2018)
- [An efficient implementable inexact entropic proximal point algorithm for a class of linear programming problems](https://mlsanity.com/paper/2011.14312/) (2020)
- [A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow](https://mlsanity.com/paper/2110.11991/) (2021)
- [QSDPNAL: A two-phase augmented Lagrangian method for convex quadratic semidefinite programming](https://mlsanity.com/paper/1512.08872/) (2015)
- [QPPAL: A two-phase proximal augmented Lagrangian method for high dimensional convex quadratic programming problems](https://mlsanity.com/paper/2103.13108/) (2021)
- [A Corrected Inexact Proximal Augmented Lagrangian Method with a Relative Error Criterion for a Class of Group-quadratic Regularized Optimal Transport Problems](https://mlsanity.com/paper/2311.01976/) (2023)
- [A Sparse Smoothing Newton Method for Solving Discrete Optimal Transport Problems](https://mlsanity.com/paper/2311.06448/) (2023)
- [Fast algorithms for large scale generalized distance weighted discrimination](https://mlsanity.com/paper/1604.05473/) (2016)
- [An Interior-Point-Inspired algorithm for Linear Programs arising in Discrete Optimal Transport](https://mlsanity.com/paper/2206.11009/) (2022)

## Built on by (most cited first, 2 of 2)

- [PINS: Proximal Iterations with Sparse Newton and Sinkhorn for Optimal Transport](https://mlsanity.com/paper/2502.03749/) (2025)
- [A Provably Convergent and Practical Algorithm for Gromov--Wasserstein Optimal Transport](https://mlsanity.com/paper/2605.04175/) (2026)

## Abstract (arXiv)

Learning to Optimize (L2O) approaches, including algorithm unrolling, plug-and-play methods, and hyperparameter learning, have garnered significant attention and have been successfully applied to the Alternating Direction Method of Multipliers (ADMM) and its variants. However, the natural extension of L2O to multi-block ADMM-type methods remains largely unexplored. Such an extension is critical, as multi-block methods leverage the separable structure of optimization problems, offering substantial reductions in per-iteration complexity. Given that classical multi-block ADMM does not guarantee convergence, the Majorized Proximal Augmented Lagrangian Method (MPALM), which shares a similar form with multi-block ADMM and ensures convergence, is more suitable in this setting. Despite its theoretical advantages, MPALM's performance is highly sensitive to the choice of penalty parameters. To address this limitation, we propose a novel L2O approach that adaptively selects this hyperparameter using supervised learning. We demonstrate the versatility and effectiveness of our method by applying it to the Lasso problem and the optimal transport problem. Our numerical results show that the proposed framework outperforms popular alternatives. Given its applicability to generic linearly constrained composite optimization problems, this work opens the door to a wide range of potential real-world applications.