# Data-Enabled Policy Optimization for Direct Adaptive Learning of the LQR

arXiv: https://arxiv.org/abs/2401.14871 · HTML: https://mlsanity.com/paper/2401.14871/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2024.

## Built on by (most cited first, 9 of 9)

- [Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches](https://mlsanity.com/paper/2505.03706/) (2025)
- [NeoRL: Efficient Exploration for Nonepisodic RL](https://mlsanity.com/paper/2406.01175/) (2024)
- [Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence](https://mlsanity.com/paper/2406.03734/) (2024)
- [Gradient Dominance in the Linear Quadratic Regulator: A Unified Analysis for Continuous-Time and Discrete-Time Systems](https://mlsanity.com/paper/2602.22577/) (2026)
- [Regret Analysis of Policy Optimization over Submanifolds for Linearly Constrained Online LQG](https://mlsanity.com/paper/2403.08553/) (2024)
- [Interpolation Conditions for Data Consistency and Prediction in Noisy Linear Systems](https://mlsanity.com/paper/2504.08484/) (2025)
- [Adaptive Inverse Reinforcement Learning with Online Off-Policy Data Collection](https://mlsanity.com/paper/2511.15171/) (2025)
- [Adaptive Control of Unknown Linear Switched Systems via Policy Gradient Methods](https://mlsanity.com/paper/2601.03016/) (2026)
- [Data-Driven Continuous-Time Linear Quadratic Regulator via Closed-Loop and Reinforcement Learning Parameterizations](https://mlsanity.com/paper/2604.27922/) (2026)

## Abstract (arXiv)

Direct data-driven design methods for the linear quadratic regulator (LQR) mainly use offline or episodic data batches, and their online adaptation has been acknowledged as an open problem. In this paper, we propose a direct adaptive method to learn the LQR from online closed-loop data. First, we propose a new policy parameterization based on the sample covariance to formulate a direct data-driven LQR problem, which is shown to be equivalent to the certainty-equivalence LQR with optimal non-asymptotic guarantees. Second, we design a novel data-enabled policy optimization (DeePO) method to directly update the policy, where the gradient is explicitly computed using only a batch of persistently exciting (PE) data. Third, we establish its global convergence via a projected gradient dominance property. Importantly, we efficiently use DeePO to adaptively learn the LQR by performing only one-step projected gradient descent per sample of the closed-loop system, which also leads to an explicit recursive update of the policy. Under PE inputs and for bounded noise, we show that the average regret of the LQR cost is upper-bounded by two terms signifying a sublinear decrease in time O(1/√T) plus a bias scaling inversely with signal-to-noise ratio (SNR), which are independent of the noise statistics. Finally, we perform simulations to validate the theoretical results and demonstrate the computational and sample efficiency of our method.