# Optimal Cost Design for Model Predictive Control

arXiv: https://arxiv.org/abs/2104.11353 · HTML: https://mlsanity.com/paper/2104.11353/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 30 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control](https://mlsanity.com/paper/1811.01848/) (2018)
- [Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC-Guided Policy Search](https://mlsanity.com/paper/1509.06791/) (2015)
- [Automatic LQR Tuning Based on Gaussian Process Global Optimization](https://mlsanity.com/paper/1605.01950/) (2016)
- [Deep Value Model Predictive Control](https://mlsanity.com/paper/1910.03358/) (2019)
- [Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robot](https://mlsanity.com/paper/2003.03200/) (2020)
- [ABC-LMPC: Safe Sample-Based Learning MPC for Stochastic Nonlinear Dynamical Systems with Adjustable Boundary Conditions](https://mlsanity.com/paper/2003.01410/) (2020)
- [Dynamically Switching Human Prediction Models for Efficient Planning](https://mlsanity.com/paper/2103.07815/) (2021)

## Built on by (most cited first, 13 of 13)

- [Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care](https://mlsanity.com/paper/2208.07406/) (2022)
- [Deep Model Predictive Optimization](https://mlsanity.com/paper/2310.04590/) (2023)
- [Real-time goal recognition using approximations in Euclidean space](https://mlsanity.com/paper/2307.07876/) (2023)
- [Traj-LeWM: Path-Aware World-Model Planning via Latent Trajectory Cost](https://mlsanity.com/paper/2608.14125/) (2026)
- [Bounding Distributional Shifts in World Modeling through Novelty Detection](https://mlsanity.com/paper/2508.06096/) (2025)
- [Zeroth-Order Actor-Critic: An Evolutionary Framework for Sequential Decision Problems](https://mlsanity.com/paper/2201.12518/) (2022)
- [Reinforcement Teaching](https://mlsanity.com/paper/2204.11897/) (2022)
- [Performance-Driven Controller Tuning via Derivative-Free Reinforcement Learning](https://mlsanity.com/paper/2209.04854/) (2022)
- [Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion](https://mlsanity.com/paper/2307.07752/) (2023)
- [Real-Time Model Checking for Closed-Loop Robot Reactive Planning](https://mlsanity.com/paper/2508.19186/) (2025)
- [Performance Index Shaping for Closed-loop Optimal Control](https://mlsanity.com/paper/2510.10202/) (2025)
- [Online Goal Recognition using Path Signature and Dynamic Time Warping](https://mlsanity.com/paper/2605.07736/) (2026)
- [Adaptive Smooth Tchebycheff Attention for Multi-Objective Policy Optimization](https://mlsanity.com/paper/2605.12771/) (2026)

## Abstract (arXiv)

Many robotics domains use some form of nonconvex model predictive control (MPC) for planning, which sets a reduced time horizon, performs trajectory optimization, and replans at every step. The actual task typically requires a much longer horizon than is computationally tractable, and is specified via a cost function that cumulates over that full horizon. For instance, an autonomous car may have a cost function that makes a desired trade-off between efficiency, safety, and obeying traffic laws. In this work, we challenge the common assumption that the cost we optimize using MPC should be the same as the ground truth cost for the task (plus a terminal cost). MPC solvers can suffer from short planning horizons, local optima, incorrect dynamics models, and, importantly, fail to account for future replanning ability. Thus, we propose that in many tasks it could be beneficial to purposefully choose a different cost function for MPC to optimize: one that results in the MPC rollout having low ground truth cost, rather than the MPC planned trajectory. We formalize this as an optimal cost design problem, and propose a zeroth-order optimization-based approach that enables us to design optimal costs for an MPC planning robot in continuous MDPs. We test our approach in an autonomous driving domain where we find costs different from the ground truth that implicitly compensate for replanning, short horizon, incorrect dynamics models, and local minima issues. As an example, the learned cost incentivizes MPC to delay its decision until later, implicitly accounting for the fact that it will get more information in the future and be able to make a better decision. Code and videos available at https://sites.google.com/berkeley.edu/ocd-mpc/.