# Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Control

arXiv: https://arxiv.org/abs/2311.04303 · HTML: https://mlsanity.com/paper/2311.04303/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning](https://mlsanity.com/paper/1803.11347/) (2018)
- [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [acados: a modular open-source framework for fast embedded optimal control](https://mlsanity.com/paper/1910.13753/) (2019)
- [Probabilistic model predictive safety certification for learning-based control](https://mlsanity.com/paper/1906.10417/) (2019)
- [Stochastic data-driven model predictive control using Gaussian processes](https://mlsanity.com/paper/1908.01786/) (2019)
- [Reinforcement Learning for Mixed-Integer Problems Based on MPC](https://mlsanity.com/paper/2004.01430/) (2020)
- [EDGAR: An Autonomous Driving Research Platform -- From Feature Development to Real-World Application](https://mlsanity.com/paper/2309.15492/) (2023)
- [Reinforcement Learning of the Prediction Horizon in Model Predictive Control](https://mlsanity.com/paper/2102.11122/) (2021)
- [Sample-Based Learning Model Predictive Control for Linear Uncertain Systems](https://mlsanity.com/paper/1904.06432/) (2019)
- [MPC-based Reinforcement Learning for a Simplified Freight Mission of Autonomous Surface Vehicles](https://mlsanity.com/paper/2106.08634/) (2021)
- [Optimization of the Model Predictive Control Meta-Parameters Through Reinforcement Learning](https://mlsanity.com/paper/2111.04146/) (2021)
- [A Stochastic Nonlinear Model Predictive Control with an Uncertainty Propagation Horizon for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2310.18753/) (2023)
- [Combining Gaussian processes and polynomial chaos expansions for stochastic nonlinear model predictive control](https://mlsanity.com/paper/2103.05441/) (2021)

## Built on by (most cited first, 6 of 6)

- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [A Safe Reinforcement Learning driven Weights-varying Model Predictive Control for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2402.02624/) (2024)
- [Generalized Kalman filter based temporal difference reinforcement learning](https://mlsanity.com/paper/2607.20010/) (2026)
- [Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping](https://mlsanity.com/paper/2607.23930/) (2026)
- [SelectLight: Learning to Select Signal Plans Generated by Distributed Model Predictive Control for Urban Traffic Networks](https://mlsanity.com/paper/2608.12256/) (2026)
- [Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC](https://mlsanity.com/paper/2609.01061/) (2026)

## Abstract (arXiv)

In this paper, we present a Deep Reinforcement Learning (RL)-driven Adaptive Stochastic Nonlinear Model Predictive Control (SNMPC) to optimize uncertainty handling, constraints robustification, feasibility, and closed-loop performance. To this end, we conceive an RL agent to proactively anticipate upcoming control tasks and to dynamically determine the most suitable combination of key SNMPC parameters - foremost the robustification factor kappa and the Uncertainty Propagation Horizon (UPH) Tu. We analyze the trained RL agent's decision-making process and highlight its ability to learn context-dependent optimal parameters. One key finding is that adapting the constraints robustification factor with the learned policy reduces conservatism and improves closed-loop performance while adapting UPH renders previously infeasible SNMPC problems feasible when faced with severe disturbances. We showcase the enhanced robustness and feasibility of our Adaptive SNMPC (aSNMPC) through the real-time motion control task of an autonomous passenger vehicle to follow an optimal race line when confronted with significant time-variant disturbances. Experimental findings demonstrate that our look-ahead RL-driven aSNMPC outperforms its Static SNMPC (sSNMPC) counterpart in minimizing the lateral deviation both with accurate and inaccurate disturbance assumptions and even when driving in previously unexplored environments.