mlsanityData to 14 Oct 2026

Convergence of Policy Gradient for Stochastic Linear-Quadratic Control Problem in Infinite Horizon

arXiv 2404.11382 ↗First posted 17 Apr 2024math.OCcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

With the outstanding performance of policy gradient (PG) method in the reinforcement learning field, the convergence theory of it has aroused more and more interest recently. Meanwhile, the significant importance and abundant theoretical researches make the stochastic linear quadratic (SLQ) control problem a starting point for studying PG in model-based learning setting. In this paper, we study the PG method for the SLQ problem in infinite horizon and take a step towards providing rigorous guarantees for gradient methods. Although the cost functional of linear-quadratic problem is typically nonconvex, we still overcome the difficulty based on gradient domination condition and L-smoothness property, and prove exponential/linear convergence of gradient flow/descent algorithm.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 42 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

17 of 42 references.

References · 17

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  3. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
  4. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  5. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  6. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 101
  7. 2022q-Learning in Continuous Timecited by 91
  8. 2021Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approachcited by 84
  9. 2020Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizoncited by 50
  10. 2019Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problemcited by 46
  11. 2019LQR through the Lens of First Order Methods: Discrete-time Casecited by 40
  12. 2020Optimizing Static Linear Feedback: Gradient Methodcited by 27
5 more
  1. 2022Convergence of policy gradient methods for finite-horizon exploratory linear-quadratic control problemscited by 25
  2. 2020Global Convergence of Policy Gradient for Linear-Quadratic Mean-Field Control/Game in Continuous Timecited by 20
  3. 2020Policy Gradient-based Algorithms for Continuous-time Linear Quadratic Controlcited by 13
  4. 2020Stochastic Linear Quadratic Optimal Control Problem: A Reinforcement Learning Methodcited by 11
  5. 2016Stochastic Linear Quadratic Optimal Control Problems in Infinite Horizoncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026(Cheap) Stochastic Policy Gradient Converges with High Probability for Linear Quadratic Regulator
  2. 2024Full error analysis of policy gradient learning algorithms for exploratory linear quadratic mean-field control problem in continuous time with common noise
  3. 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
  4. 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression
  5. 2025Global Convergence of Policy Gradient for Entropy Regularized Linear-Quadratic Control with Multiplicative Noise
  6. 2025Second-Order Policy Gradient Methods for the Linear Quadratic Regulator
  7. 2025On the Gradient Domination of the LQG Problem
  8. 2025Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches