# Direct Data-Driven Discounted Infinite Horizon Linear Quadratic Regulator with Robustness Guarantees

arXiv: https://arxiv.org/abs/2409.10703 · HTML: https://mlsanity.com/paper/2409.10703/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 48 references that are in the map.
- First posted on arXiv 16 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 9 of 9)

- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [LQR through the Lens of First Order Methods: Discrete-time Case](https://mlsanity.com/paper/1907.08921/) (2019)
- [Data-driven distributionally robust LQR with multiplicative noise](https://mlsanity.com/paper/1912.09990/) (2019)
- [Learning robust control for LQR systems with multiplicative noise via policy gradient](https://mlsanity.com/paper/1905.13547/) (2019)
- [Learning Stabilizing Controllers for Unstable Linear Quadratic Regulators from a Single Trajectory](https://mlsanity.com/paper/2006.11022/) (2020)
- [Differentiable Robust LQR Layers](https://mlsanity.com/paper/2106.05535/) (2021)
- [Provably stable learning control of linear dynamics with multiplicative noise](https://mlsanity.com/paper/2207.06062/) (2022)
- [Policy Evaluation in Distributional LQR](https://mlsanity.com/paper/2303.13657/) (2023)

## Built on by (most cited first, 1 of 1)

- [Direct Data Driven Control Using Noisy Measurements](https://mlsanity.com/paper/2505.06407/) (2025)

## Abstract (arXiv)

This paper presents a one-shot learning approach with performance and robustness guarantees for the linear quadratic regulator (LQR) control of stochastic linear systems. Even though data-based LQR control has been widely considered, existing results suffer either from data hungriness due to the inherently iterative nature of the optimization formulation (e.g., value learning or policy gradient reinforcement learning algorithms) or from a lack of robustness guarantees in one-shot non-iterative algorithms. To avoid data hungriness while ensuing robustness guarantees, an adaptive dynamic programming formalization of the LQR is presented that relies on solving a Bellman inequality. The control gain and the value function are directly learned by using a control-oriented approach that characterizes the closed-loop system using data and a decision variable from which the control is obtained. This closed-loop characterization is noise-dependent. The effect of the closed-loop system noise on the Bellman inequality is considered to ensure both robust stability and suboptimal performance despite ignoring the measurement noise. To ensure robust stability, it is shown that this system characterization leads to a closed-loop system with multiplicative and additive noise, enabling the application of distributional robust control techniques. The analysis of the suboptimality gap reveals that robustness can be achieved without the need for regularization or parameter tuning. The simulation results on the active car suspension problem demonstrate the superiority of the proposed method in terms of robustness and performance gap compared to existing methods.