# On Architectures for Combining Reinforcement Learning and Model Predictive Control with Runtime Improvements

arXiv: https://arxiv.org/abs/2510.03354 · HTML: https://mlsanity.com/paper/2510.03354/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 2 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 8 of 8)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [Efficient representation and approximation of model predictive control laws via deep learning](https://mlsanity.com/paper/1806.10644/) (2018)
- [Learning for MPC with Stability & Safety Guarantees](https://mlsanity.com/paper/2012.07369/) (2020)
- [Recurrent Neural Network-based Model Predictive Control for Continuous Pharmaceutical Manufacturing](https://mlsanity.com/paper/1807.09556/) (2018)
- [Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic Programming](https://mlsanity.com/paper/2406.00592/) (2024)
- [Adaptive MPC for Autonomous Lane Keeping](https://mlsanity.com/paper/1806.04335/) (2018)
- [Robust Tracking Control for Nonlinear Systems: Performance optimization via extremum seeking](https://mlsanity.com/paper/2304.00138/) (2023)

## Abstract (arXiv)

Model Predictive Control (MPC) faces computational demands and performance degradation from model inaccuracies. We propose two architectures combining Neural Network-approximated MPC (NNMPC) with Reinforcement Learning (RL). The first, Warm Start RL, initializes the RL actor with pre-trained NNMPC weights. The second, RLMPC, uses RL to generate corrective residuals for NNMPC outputs. We introduce a downsampling method reducing NNMPC input dimensions while maintaining performance. Evaluated on a rotary inverted pendulum, both architectures demonstrate runtime reductions exceeding 99% compared to traditional MPC while improving tracking performance under model uncertainties, with RL+MPC achieving 11-40% cost reduction depending on reference amplitude.