# Reinforcement Learning based on MPC/MHE for Unmodeled and Partially Observable Dynamics

arXiv: https://arxiv.org/abs/2103.11871 · HTML: https://mlsanity.com/paper/2103.11871/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2021.

## Built on by (most cited first, 5 of 5)

- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [MPC-based Reinforcement Learning for Economic Problems with Application to Battery Storage](https://mlsanity.com/paper/2104.02411/) (2021)
- [Optimization of the Model Predictive Control Meta-Parameters Through Reinforcement Learning](https://mlsanity.com/paper/2111.04146/) (2021)
- [Neural Moving Horizon Estimation for Robust Flight Control](https://mlsanity.com/paper/2206.10397/) (2022)
- [Approximate Robust NMPC using Reinforcement Learning](https://mlsanity.com/paper/2104.02743/) (2021)

## Abstract (arXiv)

This paper proposes an observer-based framework for solving Partially Observable Markov Decision Processes (POMDPs) when an accurate model is not available. We first propose to use a Moving Horizon Estimation-Model Predictive Control (MHE-MPC) scheme in order to provide a policy for the POMDP problem, where the full state of the real process is not measured and necessarily known. We propose to parameterize both MPC and MHE formulations, where certain adjustable parameters are regarded for tuning the policy. In this paper, for the sake of tackling the unmodeled and partially observable dynamics, we leverage the Reinforcement Learning (RL) to tune the parameters of MPC and MHE schemes jointly, with the closed-loop performance of the policy as a goal rather than model fitting or the MHE performance. Illustrations show that the proposed approach can effectively increase the performance of close-loop control of systems formulated as POMDPs.