# Reinforcement Learning Compensated Model Predictive Control for Off-road Driving on Unknown Deformable Terrain

arXiv: https://arxiv.org/abs/2408.09253 · HTML: https://mlsanity.com/paper/2408.09253/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 17 Aug 2024.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)
- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Residual Policy Learning](https://mlsanity.com/paper/1812.06298/) (2018)
- [A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems](https://mlsanity.com/paper/1705.01292/) (2017)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [acados: a modular open-source framework for fast embedded optimal control](https://mlsanity.com/paper/1910.13753/) (2019)
- [Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms](https://mlsanity.com/paper/2203.07747/) (2022)
- [Learning Model Predictive Control for iterative tasks. A Data-Driven Control Framework](https://mlsanity.com/paper/1609.01387/) (2016)
- [Blending MPC & Value Function Approximation for Efficient Reinforcement Learning](https://mlsanity.com/paper/2012.05909/) (2020)
- [RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion](https://mlsanity.com/paper/2305.17842/) (2023)
- [A Data-Driven Automatic Tuning Method for MPC under Uncertainty using Constrained Bayesian Optimization](https://mlsanity.com/paper/2011.11841/) (2020)
- [A Neural Network Architecture to Learn Explicit MPC Controllers from Data](https://mlsanity.com/paper/1911.10789/) (2019)
- [Safe and Near-Optimal Policy Learning for Model Predictive Control using Primal-Dual Neural Networks](https://mlsanity.com/paper/1906.08257/) (2019)
- [Adaptive Robust Model Predictive Control with Matched and Unmatched Uncertainty](https://mlsanity.com/paper/2104.08261/) (2021)
- [Deep Model Predictive Optimization](https://mlsanity.com/paper/2310.04590/) (2023)
- [A Safe Reinforcement Learning driven Weights-varying Model Predictive Control for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2402.02624/) (2024)
- [Adaptive MPC under Time Varying Uncertainty: Robust and Stochastic](https://mlsanity.com/paper/1909.13473/) (2019)
- [Model Predictive Control via On-Policy Imitation Learning](https://mlsanity.com/paper/2210.09206/) (2022)
- [Online terrain estimation for autonomous vehicles on deformable terrains](https://mlsanity.com/paper/1908.00130/) (2019)
- [Adaptive tracking MPC for nonlinear systems via online linear system identification](https://mlsanity.com/paper/2405.09852/) (2024)

## Built on by (most cited first, 1 of 1)

- [Actor-Critic Cooperative Compensation to Model Predictive Control for Off-Road Autonomous Vehicles Under Unknown Dynamics](https://mlsanity.com/paper/2503.00577/) (2025)

## Abstract (arXiv)

This study presents an Actor-Critic reinforcement learning Compensated Model Predictive Controller (AC2MPC) designed for high-speed, off-road autonomous driving on deformable terrains. Addressing the difficulty of modeling unknown tire-terrain interaction and ensuring real-time control feasibility and performance, this framework integrates deep reinforcement learning with a model predictive controller to manage unmodeled nonlinear dynamics. We evaluate the controller framework over constant and varying velocity profiles using high-fidelity simulator Project Chrono. Our findings demonstrate that our controller statistically outperforms standalone model-based and learning-based controllers over three unknown terrains that represent sandy deformable track, sandy and rocky track and cohesive clay-like deformable soil track. Despite varied and previously unseen terrain characteristics, this framework generalized well enough to track longitudinal reference speeds with the least error. Furthermore, this framework required significantly less training data compared to purely learning based controller, converging in fewer steps while delivering better performance. Even when under-trained, this controller outperformed the standalone controllers, highlighting its potential for safer and more efficient real-world deployment.