# OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression

arXiv: https://arxiv.org/abs/2105.13271 · HTML: https://mlsanity.com/paper/2105.13271/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 69 references that are in the map.
- First posted on arXiv 27 May 2021.

## Builds on (most cited first, 30 of 38)

- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Non-stationary Stochastic Optimization](https://mlsanity.com/paper/1307.5449/) (2013)
- [Stochastic model-based minimization of weakly convex functions](https://mlsanity.com/paper/1803.06523/) (2018)
- [Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems](https://mlsanity.com/paper/1704.03488/) (2017)
- [Online Optimization : Competing with Dynamic Comparators](https://mlsanity.com/paper/1501.06225/) (2015)
- [Learning Maximally Monotone Operators for Image Recovery](https://mlsanity.com/paper/2012.13247/) (2020)
- [Error bounds, quadratic growth, and linear convergence of proximal methods](https://mlsanity.com/paper/1602.06661/) (2016)
- [Regularization by Denoising via Fixed-Point Projection (RED-PRO)](https://mlsanity.com/paper/2008.00226/) (2020)
- [Stochastic Methods for Composite and Weakly Convex Optimization Problems](https://mlsanity.com/paper/1703.08570/) (2017)
- [Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes](https://mlsanity.com/paper/1603.04915/) (2016)
- [Non-asymptotic Identification of Linear Dynamical Systems Using Multiple Trajectories](https://mlsanity.com/paper/2009.00739/) (2020)
- [Smooth Strongly Convex Interpolation and Exact Worst-case Performance of First-order Methods](https://mlsanity.com/paper/1502.05666/) (2015)
- [Convergence of a Stochastic Gradient Method with Momentum for Non-Smooth Non-Convex Optimization](https://mlsanity.com/paper/2002.05466/) (2020)
- [Online Optimization with Predictions and Switching Costs: Fast Algorithms and the Fundamental Limit](https://mlsanity.com/paper/1801.07780/) (2018)
- [Globally Convergent Type-I Anderson Acceleration for Non-Smooth Fixed-Point Iterations](https://mlsanity.com/paper/1808.03971/) (2018)
- [On the Regret Analysis of Online LQR Control with Predictions](https://mlsanity.com/paper/2102.01309/) (2021)
- [Convex Calibration Dimension for Multiclass Loss Matrices](https://mlsanity.com/paper/1408.2764/) (2014)
- [Regularity as Regularization: Smooth and Strongly Convex Brenier Potentials in Optimal Transport](https://mlsanity.com/paper/1905.10812/) (2019)
- [A Computational Framework for Multivariate Convex Regression and its Variants](https://mlsanity.com/paper/1509.08165/) (2015)
- [Optimization and Learning with Information Streams: Time-varying Algorithms and Applications](https://mlsanity.com/paper/1910.08123/) (2019)
- [Operator Splitting Performance Estimation: Tight contraction factors and optimal parameter selection](https://mlsanity.com/paper/1812.00146/) (2018)
- [Derivation and Analysis of the Primal-Dual Method of Multipliers Based on Monotone Operator Theory](https://mlsanity.com/paper/1706.02654/) (2017)
- [Anderson Acceleration of Proximal Gradient Methods](https://mlsanity.com/paper/1910.08590/) (2019)
- [Regret minimization in stochastic non-convex learning via a proximal-gradient approach](https://mlsanity.com/paper/2010.06250/) (2020)
- [Data-driven nonsmooth optimization](https://mlsanity.com/paper/1808.00946/) (2018)
- [Personalized Optimization with User's Feedback](https://mlsanity.com/paper/1905.00775/) (2019)
- [Projection Methods: Swiss Army Knives for Solving Feasibility and Best Approximation Problems with Halfspaces](https://mlsanity.com/paper/1301.4506/) (2013)
- [Linear Convergence and Metric Selection for Douglas-Rachford Splitting and ADMM](https://mlsanity.com/paper/1410.8479/) (2014)
- [Unifying Lower Bounds on Prediction Dimension of Consistent Convex Surrogates](https://mlsanity.com/paper/2102.08218/) (2021)

## Built on by (most cited first, 1 of 1)

- [Neural Fixed-Point Acceleration for Convex Optimization](https://mlsanity.com/paper/2107.10254/) (2021)

## Abstract (arXiv)

This paper presents a new regularization approach -- termed OpReg-Boost -- to boost the convergence and lessen the asymptotic error of online optimization and learning algorithms. In particular, the paper considers online algorithms for optimization problems with a time-varying (weakly) convex composite cost. For a given online algorithm, OpReg-Boost learns the closest algorithmic map that yields linear convergence; to this end, the learning procedure hinges on the concept of operator regression. We show how to formalize the operator regression problem and propose a computationally-efficient Peaceman-Rachford solver that exploits a closed-form solution of simple quadratically-constrained quadratic programs (QCQPs). Simulation results showcase the superior properties of OpReg-Boost w.r.t. the more classical forward-backward algorithm, FISTA, and Anderson acceleration.