# Combining system identification with reinforcement learning-based MPC

arXiv: https://arxiv.org/abs/2004.03265 · HTML: https://mlsanity.com/paper/2004.03265/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 11 of 11)

- [Assured Learning-enabled Autonomy: A Metacognitive Reinforcement Learning Framework](https://mlsanity.com/paper/2103.12558/) (2021)
- [Quasi-Newton Iteration in Deterministic Policy Gradient](https://mlsanity.com/paper/2203.13854/) (2022)
- [MPC-based Imitation Learning for Safe and Human-like Autonomous Driving](https://mlsanity.com/paper/2206.12348/) (2022)
- [Concurrent Policy Blending and System Identification for Generalized Assistive Control](https://mlsanity.com/paper/2205.09836/) (2022)
- [Convergent NMPC-based Reinforcement Learning Using Deep Expected Sarsa and Nonlinear Temporal Difference Learning](https://mlsanity.com/paper/2502.04925/) (2025)
- [Toward 6-DOF Autonomous Underwater Vehicle Energy-Aware Position Control based on Deep Reinforcement Learning: Preliminary Results](https://mlsanity.com/paper/2502.17742/) (2025)
- [Human-aligned Safe Reinforcement Learning for Highway On-Ramp Merging in Dense Traffic](https://mlsanity.com/paper/2503.02624/) (2025)
- [Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agents](https://mlsanity.com/paper/2507.13491/) (2025)
- [Zero-Shot Function Encoder-Based Differentiable Predictive Control](https://mlsanity.com/paper/2511.05757/) (2025)
- [MPC-Guided Safe Reinforcement Learning and Lipschitz-Based Filtering for Structured Nonlinear Systems](https://mlsanity.com/paper/2512.12855/) (2025)
- [Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC](https://mlsanity.com/paper/2609.01061/) (2026)

## Abstract (arXiv)

In this paper we propose and compare methods for combining system identification (SYSID) and reinforcement learning (RL) in the context of data-driven model predictive control (MPC). Assuming a known model structure of the controlled system, and considering a parametric MPC, the proposed approach simultaneously: a) Learns the parameters of the MPC using RL in order to optimize performance, and b) fits the observed model behaviour using SYSID. Six methods that avoid conflicts between the two optimization objectives are proposed and evaluated using a simple linear system. Based on the simulation results, hierarchical, parallel projection, nullspace projection, and singular value projection achieved the best performance.