mlsanityData to 14 Oct 2026

Stability-Certified On-Policy Data-Driven LQR via Recursive Learning and Policy Gradient

arXiv 2403.05367 ↗First posted 8 Mar 2024eess.SY cs.SYcited by 1

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

In this paper, we investigate a data-driven framework to solve Linear Quadratic Regulator (LQR) problems when the dynamics is unknown, with the additional challenge of providing stability certificates for the overall learning and control scheme. Specifically, in the proposed on-policy learning framework, the control input is applied to the actual (unknown) linear system while iteratively optimized. We propose a learning and control procedure, termed Relearn LQR, that combines a recursive least squares method with a direct policy search based on the gradient method. The resulting scheme is analyzed by modeling it as a feedback-interconnected nonlinear dynamical system. A Lyapunov-based approach, exploiting averaging and timescale separation theories for nonlinear systems, allows us to provide formal stability guarantees for the whole interconnected scheme. The effectiveness of the proposed strategy is corroborated by numerical simulations, where Relearn LQR is deployed on an aircraft control problem, with both static and drifting parameters.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 64 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

25 of 64 references.

References · 25

  1. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  2. 2017On the Sample Complexity of the Linear Quadratic Regulatorcited by 159
  3. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  4. 2020Naive Exploration is Optimal for Online LQRcited by 116
  5. 2019Learning Linear-Quadratic Regulators Efficiently with only √T Regretcited by 70
  6. 2019Formulas for Data-driven Control: Stabilization, Optimality and Robustnesscited by 47
  7. 2019Policy Optimization for H2 Linear Control with H_∞ Robustness Guarantee: Implicit Regularization and Global Convergencecited by 46
  8. 2019Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problemcited by 46
  9. 2019LQR through the Lens of First Order Methods: Discrete-time Casecited by 40
  10. 2020Logarithmic Regret for Learning Linear Quadratic Regulators Efficientlycited by 35
  11. 2018Safely Learning to Control the Constrained Linear Quadratic Regulatorcited by 33
  12. 2019Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulatorcited by 23
13 more
  1. 2019Data informativity: a new perspective on data-driven analysis and controlcited by 19
  2. 2020Robust Policy Iteration for Continuous-time Linear Quadratic Regulationcited by 15
  3. 2020From noisy data to feedback controllers: non-conservative design via a matrix S-lemmacited by 11
  4. 2021Bridging direct & indirect data-driven control formulations via regularizations and relaxationscited by 11
  5. 2022How are policy gradient methods affected by the limits of control?cited by 10
  6. 2021Efficient Off-Policy Q-Learning for Data-Based Discrete-Time LQR Problemscited by 9
  7. 2020Low-complexity Learning of Linear Quadratic Regulators from Noisy Datacited by 8
  8. 2019Robust data-driven state-feedback designcited by 7
  9. 2019On Topological Properties of the Set of Stabilizing Feedback Gainscited by 5
  10. 2021On the Certainty-Equivalence Approach to Direct Data-Driven LQR Designcited by 5
  11. 2021Online learning of data-driven controllers for unknown switched linear systemscited by 3
  12. 2019Data-driven Linear Quadratic Regulation via Semidefinite Programmingcited by 2
  13. 2019Structured exploration in the finite horizon linear quadratic dual control problemcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024A Moreau Envelope Approach for LQR Meta-Policy Estimationcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Data-Driven Continuous-Time Linear Quadratic Regulator via Closed-Loop and Reinforcement Learning Parameterizations
  2. 2024MR-ARL: Model Reference Adaptive Reinforcement Learning for Robustly Stable On-Policy Data-Driven LQR
  3. 2025Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches
  4. 2025On the Gradient Domination of the LQG Problem
  5. 2025A Fully Data-Driven Value Iteration for Stochastic LQR: Convergence, Robustness and Stability
  6. 2026Global Convergence of Policy Gradient Methods for ReLU Controllers in Linear Quadratic Regulation
  7. 2025Second-Order Policy Gradient Methods for the Linear Quadratic Regulator
  8. 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression