# A Safe Reinforcement Learning driven Weights-varying Model Predictive Control for Autonomous Vehicle Motion Control

arXiv: https://arxiv.org/abs/2402.02624 · HTML: https://mlsanity.com/paper/2402.02624/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 4 Feb 2024.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)
- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Safe Controller Optimization for Quadrotors with Gaussian Processes](https://mlsanity.com/paper/1509.01066/) (2015)
- [Dirichlet-based Gaussian Processes for Large-scale Calibrated Classification](https://mlsanity.com/paper/1805.10915/) (2018)
- [Performance-oriented model learning for data-driven MPC design](https://mlsanity.com/paper/1904.10839/) (2019)
- [A Data-Driven Automatic Tuning Method for MPC under Uncertainty using Constrained Bayesian Optimization](https://mlsanity.com/paper/2011.11841/) (2020)
- [EDGAR: An Autonomous Driving Research Platform -- From Feature Development to Real-World Application](https://mlsanity.com/paper/2309.15492/) (2023)
- [Reinforcement Learning of the Prediction Horizon in Model Predictive Control](https://mlsanity.com/paper/2102.11122/) (2021)
- [Performance-based Trajectory Optimization for Path Following Control Using Bayesian Optimization](https://mlsanity.com/paper/2103.15416/) (2021)
- [Optimization of the Model Predictive Control Meta-Parameters Through Reinforcement Learning](https://mlsanity.com/paper/2111.04146/) (2021)
- [Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2311.04303/) (2023)
- [A Stochastic Nonlinear Model Predictive Control with an Uncertainty Propagation Horizon for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2310.18753/) (2023)
- [Multi-Objective Optimization of a Path-following MPC for Vehicle Guidance: A Bayesian Optimization Approach](https://mlsanity.com/paper/2104.03773/) (2021)
- [R2NMPC: A Real-Time Reduced Robustified Nonlinear Model Predictive Control with Ellipsoidal Uncertainty Sets for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2311.06420/) (2023)

## Built on by (most cited first, 7 of 7)

- [CACTO-BIC: Scalable Actor-Critic Learning via Biased Sampling and GPU-Accelerated Trajectory Optimization](https://mlsanity.com/paper/2602.19699/) (2026)
- [Reinforcement Learning Compensated Model Predictive Control for Off-road Driving on Unknown Deformable Terrain](https://mlsanity.com/paper/2408.09253/) (2024)
- [Safe Reinforcement Learning with a Predictive Safety Filter for Motion Planning and Control: A Drifting Vehicle Example](https://mlsanity.com/paper/2506.22894/) (2025)
- [Reinforcement Learning-based Dynamic Adaptation for Sampling-Based Motion Planning in Agile Autonomous Driving](https://mlsanity.com/paper/2510.10567/) (2025)
- [Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping](https://mlsanity.com/paper/2607.23930/) (2026)
- [SelectLight: Learning to Select Signal Plans Generated by Distributed Model Predictive Control for Urban Traffic Networks](https://mlsanity.com/paper/2608.12256/) (2026)
- [Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC](https://mlsanity.com/paper/2609.01061/) (2026)

## Abstract (arXiv)

Determining the optimal cost function parameters of Model Predictive Control (MPC) to optimize multiple control objectives is a challenging and time-consuming task. Multiobjective Bayesian Optimization (BO) techniques solve this problem by determining a Pareto optimal parameter set for an MPC with static weights. However, a single parameter set may not deliver the most optimal closed-loop control performance when the context of the MPC operating conditions changes during its operation, urging the need to adapt the cost function weights at runtime. Deep Reinforcement Learning (RL) algorithms can automatically learn context-dependent optimal parameter sets and dynamically adapt for a Weightsvarying MPC (WMPC). However, learning cost function weights from scratch in a continuous action space may lead to unsafe operating states. To solve this, we propose a novel approach limiting the RL actions within a safe learning space representing a catalog of pre-optimized BO Pareto-optimal weight sets. We conceive a RL agent not to learn in a continuous space but to proactively anticipate upcoming control tasks and to choose the most optimal discrete actions, each corresponding to a single set of Pareto optimal weights, context-dependent. Hence, even an untrained RL agent guarantees a safe and optimal performance. Experimental results demonstrate that an untrained RL-WMPC shows Pareto-optimal closed-loop behavior and training the RL-WMPC helps exhibit a performance beyond the Pareto-front.