mlsanityData to 14 Oct 2026

On the Robustness of Derivative-free Methods for Linear Quadratic Regulator

arXiv 2506.12596 ↗First posted 14 Jun 2025math.OCcited by 1

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Policy optimization has drawn increasing attention in reinforcement learning, particularly in the context of derivative-free methods for linear quadratic regulator (LQR) problems with unknown dynamics. This paper focuses on characterizing the robustness of derivative-free methods for solving an infinite-horizon LQR problem. To be specific, we estimate policy gradients by cost values, and study the effect of perturbations on the estimations, where the perturbations may arise from function approximations, measurement noises, etc. We show that under sufficiently small perturbations, the derivative-free methods converge to any pre-specified neighborhood of the optimal policy. Furthermore, we establish explicit bounds on the perturbations, and provide the sample complexity for the perturbed derivative-free methods.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 19 of its 35 references that are in the map. First posted on arXiv 14 Jun 2025.

Builds on

19 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Distributed Policy Gradient for Linear Quadratic Networked Control with Limited Communication Range5 Mar

Other references · 18

  1. 2015Trust Region Policy Optimizationcited by 2,580
  2. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 724
  3. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  4. 2017On the Sample Complexity of the Linear Quadratic Regulatorcited by 159
  5. 2015An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedbackcited by 111
  6. 2018Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systemscited by 67
  7. 2019Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problemcited by 46
  8. 2019Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Gamescited by 42
  9. 2019Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approachcited by 38
  10. 2020Optimizing Static Linear Feedback: Gradient Methodcited by 27
  11. 2021Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRscited by 16
  12. 2020Robust Policy Iteration for Continuous-time Linear Quadratic Regulationcited by 15
6 more
  1. 2016A Decentralized Energy-Optimal Control Framework for Connected Automated Vehicles at Signal-Free Intersectionscited by 11
  2. 2021Separation of Learning and Control for Cyber-Physical Systemscited by 10
  3. 2021Remarks on input to state stability of perturbed gradient flows, motivated by model-free feedback control learningcited by 8
  4. 2021A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadwayscited by 7
  5. 2022Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Controlcited by 7
  6. 2023Small-Disturbance Input-to-State Stability of Perturbed Gradient Flows: Applications to LQR Problemcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026A Stochastic Gradient Descent Approach to Design Policy Gradient Methods for LQRcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression
  2. 2024Sample Complexity of the Linear Quadratic Regulator: A Reinforcement Learning Lens
  3. 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
  4. 2026(Cheap) Stochastic Policy Gradient Converges with High Probability for Linear Quadratic Regulator
  5. 2025Bridging Continuous-time LQR and Reinforcement Learning via Gradient Flow of the Bellman Error
  6. 2025Second-Order Policy Gradient Methods for the Linear Quadratic Regulator
  7. 2024Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence
  8. 2025Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming