mlsanityData to 7 Oct 2026

Regret Analysis of Learning-Based MPC with Partially-Unknown Cost Function

arXiv 2108.02307 ↗First posted 4 Aug 2021math.OC cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

The exploration/exploitation trade-off is an inherent challenge in data-driven adaptive control. Though this trade-off has been studied for multi-armed bandits (MAB's) and reinforcement learning for linear systems; it is less well-studied for learning-based control of nonlinear systems. A significant theoretical challenge in the nonlinear setting is that there is no explicit characterization of an optimal controller for a given set of cost and system parameters. We propose the use of a finite-horizon oracle controller with full knowledge of parameters as a reasonable surrogate to optimal controller. This allows us to develop policies in the context of learning-based MPC and MAB's and conduct a control-theoretic analysis using techniques from MPC- and optimization-theory to show these policies achieve low regret with respect to this finite-horizon oracle. Our simulations exhibit the low regret of our policy on a heating, ventilation, and air-conditioning model with partially-unknown cost function.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 107 references that are in the map. First posted on arXiv 4 Aug 2021.

Builds on

23 of 107 references.

References · 23

  1. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 211
  2. 2018Learning-based Model Predictive Control for Safe Explorationcited by 116
  3. 2020Naive Exploration is Optimal for Online LQRcited by 111
  4. 2020Information Theoretic Regret Bounds for Online Nonlinear Controlcited by 87
  5. 2019Safe Reinforcement Learning Using Robust MPCcited by 72
  6. 2019Learning Linear-Quadratic Regulators Efficiently with only √T Regretcited by 67
  7. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 55
  8. 2013Dynamical Models and Tracking Regret in Online Convex Programmingcited by 42
  9. 2015Inverse Optimization with Noisy Datacited by 34
  10. 2020Regret Bounds for Adaptive Nonlinear Controlcited by 25
  11. 2017Non-Stationary Bandits with Habituation and Recovery Dynamicscited by 23
  12. 2013Concentration in unbounded metric spaces and algorithmic stabilitycited by 21
11 more
  1. 2015Multi-armed Bandit Problem with Known Trendcited by 17
  2. 2020Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robotcited by 15
  3. 2020Reinforcement Learning for Mixed-Integer Problems Based on MPCcited by 11
  4. 2018Model-Free Linear Quadratic Control via Reduction to Expert Predictioncited by 10
  5. 2015Dual Control for Approximate Bayesian Reinforcement Learningcited by 9
  6. 2019Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Modelscited by 7
  7. 2019Boosting for Control of Dynamical Systemscited by 7
  8. 2020Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Settingcited by 4
  9. 2019An integrated model predictive control approach for optimal HVAC and energy storage operation in large-scale buildingscited by 1
  10. 2019Linear Model-Predictive Controller (LMPC) for Building's Heating Ventilation and Air Conditioning (HVAC) Systemcited by 1
  11. 2020Stochastic Model Predictive Control for Central HVAC Plantscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 1; 2025: 0; 2026: 1.

5 papers.

  1. 2023Model Based Reinforcement Learning for Personalized Heparin Dosingcited by 5
  2. 2023Non-asymptotic System Identification for Linear Systems with Nonlinear Policiescited by 5
  3. 2023Suboptimality analysis of receding horizon quadratic control with unknown linear systems and its applications in learning-based controlcited by 1
  4. 2024Reinforcement Learning for a Discrete-Time Linear-Quadratic Control Problem with an Applicationcited by 0
  5. 2026Hierarchical Decision Making with Structured Policies: A Principled Design via Inverse Optimizationcited by 0