# Regret Analysis of Learning-Based MPC with Partially-Unknown Cost Function

arXiv: https://arxiv.org/abs/2108.02307 · HTML: https://mlsanity.com/paper/2108.02307/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 107 references that are in the map.
- First posted on arXiv 4 Aug 2021.

## Builds on (most cited first, 23 of 23)

- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Learning-based Model Predictive Control for Safe Exploration](https://mlsanity.com/paper/1803.08287/) (2018)
- [Naive Exploration is Optimal for Online LQR](https://mlsanity.com/paper/2001.09576/) (2020)
- [Information Theoretic Regret Bounds for Online Nonlinear Control](https://mlsanity.com/paper/2006.12466/) (2020)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [Learning Linear-Quadratic Regulators Efficiently with only √T Regret](https://mlsanity.com/paper/1902.06223/) (2019)
- [Data-driven Economic NMPC using Reinforcement Learning](https://mlsanity.com/paper/1904.04152/) (2019)
- [Dynamical Models and Tracking Regret in Online Convex Programming](https://mlsanity.com/paper/1301.1254/) (2013)
- [Inverse Optimization with Noisy Data](https://mlsanity.com/paper/1507.03266/) (2015)
- [Regret Bounds for Adaptive Nonlinear Control](https://mlsanity.com/paper/2011.13101/) (2020)
- [Non-Stationary Bandits with Habituation and Recovery Dynamics](https://mlsanity.com/paper/1707.08423/) (2017)
- [Concentration in unbounded metric spaces and algorithmic stability](https://mlsanity.com/paper/1309.1007/) (2013)
- [Multi-armed Bandit Problem with Known Trend](https://mlsanity.com/paper/1508.07091/) (2015)
- [Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robot](https://mlsanity.com/paper/2003.03200/) (2020)
- [Model-Free Linear Quadratic Control via Reduction to Expert Prediction](https://mlsanity.com/paper/1804.06021/) (2018)
- [Reinforcement Learning for Mixed-Integer Problems Based on MPC](https://mlsanity.com/paper/2004.01430/) (2020)
- [Dual Control for Approximate Bayesian Reinforcement Learning](https://mlsanity.com/paper/1510.03591/) (2015)
- [Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models](https://mlsanity.com/paper/1904.01068/) (2019)
- [Boosting for Control of Dynamical Systems](https://mlsanity.com/paper/1906.08720/) (2019)
- [Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting](https://mlsanity.com/paper/2003.05999/) (2020)
- [An integrated model predictive control approach for optimal HVAC and energy storage operation in large-scale buildings](https://mlsanity.com/paper/1902.06613/) (2019)
- [Linear Model-Predictive Controller (LMPC) for Building's Heating Ventilation and Air Conditioning (HVAC) System](https://mlsanity.com/paper/1906.00352/) (2019)
- [Stochastic Model Predictive Control for Central HVAC Plants](https://mlsanity.com/paper/2002.10065/) (2020)

## Built on by (most cited first, 5 of 5)

- [Model Based Reinforcement Learning for Personalized Heparin Dosing](https://mlsanity.com/paper/2304.10000/) (2023)
- [Non-asymptotic System Identification for Linear Systems with Nonlinear Policies](https://mlsanity.com/paper/2306.10369/) (2023)
- [Suboptimality analysis of receding horizon quadratic control with unknown linear systems and its applications in learning-based control](https://mlsanity.com/paper/2301.07876/) (2023)
- [Reinforcement Learning for a Discrete-Time Linear-Quadratic Control Problem with an Application](https://mlsanity.com/paper/2412.05906/) (2024)
- [Hierarchical Decision Making with Structured Policies: A Principled Design via Inverse Optimization](https://mlsanity.com/paper/2606.28764/) (2026)

## Abstract (arXiv)

The exploration/exploitation trade-off is an inherent challenge in data-driven adaptive control. Though this trade-off has been studied for multi-armed bandits (MAB's) and reinforcement learning for linear systems; it is less well-studied for learning-based control of nonlinear systems. A significant theoretical challenge in the nonlinear setting is that there is no explicit characterization of an optimal controller for a given set of cost and system parameters. We propose the use of a finite-horizon oracle controller with full knowledge of parameters as a reasonable surrogate to optimal controller. This allows us to develop policies in the context of learning-based MPC and MAB's and conduct a control-theoretic analysis using techniques from MPC- and optimization-theory to show these policies achieve low regret with respect to this finite-horizon oracle. Our simulations exhibit the low regret of our policy on a heating, ventilation, and air-conditioning model with partially-unknown cost function.