mlsanityData to 14 Oct 2026

Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression

arXiv 2512.03764 ↗First posted 3 Dec 2025eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Policy gradient algorithms are widely used in reinforcement learning and belong to the class of approximate dynamic programming methods. This paper studies two key policy gradient algorithms, the Natural Policy Gradient and the Gauss-Newton Method, for solving the Linear Quadratic Regulator (LQR) problem in unknown stochastic linear systems. The main challenge lies in obtaining a consistent gradient estimate from noisy data due to errors-in-variables in linear regression. This issue is addressed by employing a primal-dual estimation procedure. Using this novel gradient estimation scheme, the paper establishes convergence guarantees with a sample complexity of order O(1/epsilon). Theoretical results are further supported by numerical experiments, which demonstrate the effectiveness of the proposed algorithms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 19 references that are in the map. First posted on arXiv 3 Dec 2025.

Builds on

12 of 19 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches6 May
  2. 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data27 Feb
  3. 2024Sample Complexity of the Linear Quadratic Regulator: A Reinforcement Learning Lens16 Apr

Other references · 9

  1. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  2. 2018Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systemscited by 67
  3. 2020Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizoncited by 50
  4. 2017Least-Squares Temporal Difference Learning for the Linear Quadratic Regulatorcited by 30
  5. 2022Single Time-scale Actor-critic Method to Solve the Linear Quadratic Regulator with Convergence Guaranteescited by 14
  6. 2024Stochastic Online Optimization for Cyber-Physical and Robotic Systemscited by 6
  7. 2024The Role of Identification in Data-driven Policy Iteration: A System Theoretic Studycited by 3
  8. 2025Robustness of Online Identification-based Policy Iteration to Noisy Datacited by 3
  9. 2023Global Convergence of Policy Gradient Methods in Reinforcement Learning, Games and Controlcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Stochastic Gradient Descent Approach to Design Policy Gradient Methods for LQR
  2. 2026(Cheap) Stochastic Policy Gradient Converges with High Probability for Linear Quadratic Regulator
  3. 2025Second-Order Policy Gradient Methods for the Linear Quadratic Regulator
  4. 2025On the Robustness of Derivative-free Methods for Linear Quadratic Regulator
  5. 2024Full error analysis of policy gradient learning algorithms for exploratory linear quadratic mean-field control problem in continuous time with common noise
  6. 2024Policy Gradient-based Model Free Optimal LQG Control with a Probabilistic Risk Constraint
  7. 2025Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming
  8. 2025On the Gradient Domination of the LQG Problem