mlsanityData to 14 Oct 2026

MR-ARL: Model Reference Adaptive Reinforcement Learning for Robustly Stable On-Policy Data-Driven LQR

arXiv 2402.14483 ↗First posted 22 Feb 2024eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

This article introduces a novel framework for data-driven linear quadratic regulator (LQR) design. First, we introduce a reinforcement learning paradigm for on-policy data-driven LQR, where exploration and exploitation are simultaneously performed while guaranteeing robust stability of the whole closed-loop system encompassing the plant and the control/learning dynamics. Then, we propose Model Reference Adaptive Reinforcement Learning (MR-ARL), a control architecture integrating tools from reinforcement learning and model reference adaptive control. The approach stands on a variable reference model containing the currently identified value function. Then, an adaptive stabilizer is used to ensure convergence of the applied policy to the optimal one, convergence of the plant to the optimal reference model, and overall robust closed-loop stability. The proposed framework provides theoretical robustness certificates against real-world perturbations such as measurement noise, plant nonlinearities, or slowly varying parameters. The effectiveness of the proposed architecture is validated via realistic numerical simulations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 58 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

14 of 58 references.

References · 14

  1. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  2. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  3. 2019Formulas for Data-driven Control: Stabilization, Optimality and Robustnesscited by 47
  4. 2018Safely Learning to Control the Constrained Linear Quadratic Regulatorcited by 33
  5. 2019Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulatorcited by 23
  6. 2020Robust Policy Iteration for Continuous-time Linear Quadratic Regulationcited by 15
  7. 2021A Historical Perspective of Adaptive Control and Learningcited by 11
  8. 2022How are policy gradient methods affected by the limits of control?cited by 10
  9. 2021Efficient Off-Policy Q-Learning for Data-Based Discrete-Time LQR Problemscited by 9
  10. 2020Low-complexity Learning of Linear Quadratic Regulators from Noisy Datacited by 8
  11. 2021Online Policies for Real-Time Control Using MRAC-RLcited by 7
  12. 2018Data-Driven LQR Control Designcited by 5
2 more
  1. 2019Data-driven Linear Quadratic Regulation via Semidefinite Programmingcited by 2
  2. 2021Online Adaptive Optimal Control Algorithm Based on Synchronous Integral Reinforcement Learning With Explorationscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Fully Data-Driven Value Iteration for Stochastic LQR: Convergence, Robustness and Stability
  2. 2024Stability-Certified On-Policy Data-Driven LQR via Recursive Learning and Policy Gradient
  3. 2025Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches
  4. 2024Robust Model Based Reinforcement Learning Using L1 Adaptive Control
  5. 2026Data-Driven Continuous-Time Linear Quadratic Regulator via Closed-Loop and Reinforcement Learning Parameterizations
  6. 2024Direct Data-Driven Discounted Infinite Horizon Linear Quadratic Regulator with Robustness Guarantees
  7. 2025Bridging Continuous-time LQR and Reinforcement Learning via Gradient Flow of the Bellman Error
  8. 2025Certifying Stability of Reinforcement Learning Policies using Generalized Lyapunov Functions