# Online Policies for Real-Time Control Using MRAC-RL

arXiv: https://arxiv.org/abs/2103.16551 · HTML: https://mlsanity.com/paper/2103.16551/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2021.

## Built on by (most cited first, 7 of 7)

- [Improved Robustness of Deep Reinforcement Learning for Control of Time-Varying Systems by Bounded Extremum Seeking](https://mlsanity.com/paper/2510.02490/) (2025)
- [Online Algorithms and Policies Using Adaptive and Machine Learning Approaches](https://mlsanity.com/paper/2105.06577/) (2021)
- [Residual Learning and Context Encoding for Adaptive Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2406.08238/) (2024)
- [Planning for Change: Reinforcement Learning Combined with Bounded Extremum Seeking for Robotic Control under Distribution Shift](https://mlsanity.com/paper/2604.01142/) (2026)
- [Mahalanobis-Guided Latent OOD Detection for Hybrid ES-DRL Control in Time-Varying Systems](https://mlsanity.com/paper/2606.11474/) (2026)
- [MR-ARL: Model Reference Adaptive Reinforcement Learning for Robustly Stable On-Policy Data-Driven LQR](https://mlsanity.com/paper/2402.14483/) (2024)
- [Robust synchronization and policy adaptation for networked heterogeneous agents](https://mlsanity.com/paper/2409.03273/) (2024)

## Abstract (arXiv)

In this paper, we propose the Model Reference Adaptive Control & Reinforcement Learning (MRAC-RL) approach to developing online policies for systems in which modeling errors occur in real-time. Although reinforcement learning (RL) algorithms have been successfully used to develop control policies for dynamical systems, discrepancies between simulated dynamics and the true target dynamics can cause trained policies to fail to generalize and adapt appropriately when deployed in the real-world. The MRAC-RL framework generates online policies by utilizing an inner-loop adaptive controller together with a simulation-trained outer-loop RL policy. This structure allows MRAC-RL to adapt and operate effectively in a target environment, even when parametric uncertainties exists. We propose a set of novel MRAC algorithms, apply them to a class of nonlinear systems, derive the associated control laws, provide stability guarantees for the resulting closed-loop system, and show that the adaptive tracking objective is achieved. Using a simulation study of an automated quadrotor landing task, we demonstrate that the MRAC-RL approach improves upon state-of-the-art RL algorithms and techniques through the generation of online policies.