mlsanityData to 7 Oct 2026

OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression

arXiv 2105.13271 ↗First posted 27 May 2021cs.LG math.OCcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper presents a new regularization approach -- termed OpReg-Boost -- to boost the convergence and lessen the asymptotic error of online optimization and learning algorithms. In particular, the paper considers online algorithms for optimization problems with a time-varying (weakly) convex composite cost. For a given online algorithm, OpReg-Boost learns the closest algorithmic map that yields linear convergence; to this end, the learning procedure hinges on the concept of operator regression. We show how to formalize the operator regression problem and propose a computationally-efficient Peaceman-Rachford solver that exploits a closed-form solution of simple quadratically-constrained quadratic programs (QCQPs). Simulation results showcase the superior properties of OpReg-Boost w.r.t. the more classical forward-backward algorithm, FISTA, and Anderson acceleration.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 38 of its 69 references that are in the map. First posted on arXiv 27 May 2021.

Builds on

38 of 69 references.

References · 38

  1. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 290
  2. 2021Learning to Optimize: A Primer and A Benchmarkcited by 206
  3. 2013Non-stationary Stochastic Optimizationcited by 173
  4. 2018Stochastic model-based minimization of weakly convex functionscited by 157
  5. 2017Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problemscited by 107
  6. 2015Online Optimization : Competing with Dynamic Comparatorscited by 99
  7. 2020Learning Maximally Monotone Operators for Image Recoverycited by 62
  8. 2016Error bounds, quadratic growth, and linear convergence of proximal methodscited by 59
  9. 2020Regularization by Denoising via Fixed-Point Projection (RED-PRO)cited by 55
  10. 2017Stochastic Methods for Composite and Weakly Convex Optimization Problemscited by 48
  11. 2016Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processescited by 38
  12. 2020Non-asymptotic Identification of Linear Dynamical Systems Using Multiple Trajectoriescited by 38
26 more
  1. 2015Smooth Strongly Convex Interpolation and Exact Worst-case Performance of First-order Methodscited by 36
  2. 2020Convergence of a Stochastic Gradient Method with Momentum for Non-Smooth Non-Convex Optimizationcited by 32
  3. 2018Online Optimization with Predictions and Switching Costs: Fast Algorithms and the Fundamental Limitcited by 27
  4. 2018Globally Convergent Type-I Anderson Acceleration for Non-Smooth Fixed-Point Iterationscited by 24
  5. 2021On the Regret Analysis of Online LQR Control with Predictionscited by 22
  6. 2014Convex Calibration Dimension for Multiclass Loss Matricescited by 21
  7. 2019Regularity as Regularization: Smooth and Strongly Convex Brenier Potentials in Optimal Transportcited by 19
  8. 2015A Computational Framework for Multivariate Convex Regression and its Variantscited by 16
  9. 2019Optimization and Learning with Information Streams: Time-varying Algorithms and Applicationscited by 15
  10. 2018Operator Splitting Performance Estimation: Tight contraction factors and optimal parameter selectioncited by 13
  11. 2017Derivation and Analysis of the Primal-Dual Method of Multipliers Based on Monotone Operator Theorycited by 8
  12. 2019Anderson Acceleration of Proximal Gradient Methodscited by 8
  13. 2020Regret minimization in stochastic non-convex learning via a proximal-gradient approachcited by 8
  14. 2018Data-driven nonsmooth optimizationcited by 6
  15. 2019Personalized Optimization with User's Feedbackcited by 5
  16. 2013Projection Methods: Swiss Army Knives for Solving Feasibility and Best Approximation Problems with Halfspacescited by 4
  17. 2014Linear Convergence and Metric Selection for Douglas-Rachford Splitting and ADMMcited by 4
  18. 2021Unifying Lower Bounds on Prediction Dimension of Consistent Convex Surrogatescited by 4
  19. 2017Time-Varying Convex Optimization via Time-Varying Averaged Operatorscited by 3
  20. 2019Distributed generalized Nash equilibrium seeking in aggregative games on time-varying networkscited by 3
  21. 2020Socially-Aware Robot Planning via Bandit Human Feedbackcited by 2
  22. 2020Smooth Strongly Convex Regressioncited by 2
  23. 2013Sparse Recovery of Streaming Signals Using L1-Homotopycited by 1
  24. 2014A Parallel Method for Large Scale Convex Regression Problemscited by 1
  25. 2018A Semismooth Predictor Corrector Method for Real-Time Constrained Parametric Optimization with Applications in Model Predictive Controlcited by 1
  26. 2020Distributed Prediction-Correction ADMM for Time-Varying Convex Optimizationcited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1.

1 paper.

  1. 2021Neural Fixed-Point Acceleration for Convex Optimizationcited by 10