mlsanityData to 7 Oct 2026

LQR through the Lens of First Order Methods: Discrete-time Case

arXiv 1907.08921 ↗First posted 21 Jul 2019eess.SY cs.SY math.OCcited by 30

Not on the map. Why?

Abstract (arXiv)

We consider the Linear-Quadratic-Regulator (LQR) problem in terms of optimizing a real-valued matrix function over the set of feedback gains. Such a setup facilitates examining the implications of a natural initial-state independent formulation of LQR in designing first order algorithms. It is shown that this cost function is smooth and coercive, and provide an alternate means of noting its gradient dominated property. In the process, we provide a number of analytic observations on the LQR cost when directly analyzed in terms of the feedback gain. We then examine three types of well-posed flows for LQR: gradient flow, natural gradient flow and the quasi-Newton flow. The coercive property suggests that these flows admit unique solutions while gradient dominated property indicates that the corresponding Lyapunov functionals decay at an exponential rate; we also prove that these flows are exponentially stable in the sense of Lyapunov. We then discuss the forward Euler discretization of these flows, realized as gradient descent, natural gradient descent and the quasi-Newton iteration. We present stepsize criteria for gradient descent and natural gradient descent, guaranteeing that both algorithms converge linearly to the global optima. An optimal stepsize for the quasi-Newton iteration is also proposed, guaranteeing a Q-quadratic convergence rate--and in the meantime--recovering the Hewer algorithm.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 5; 2023: 8; 2024: 8; 2025: 1; 2026: 3.

30 papers.

Sort:

By area:

  1. 2023A Fisher-Rao gradient flow for entropy-regularised Markov decision processes in Polish spacescited by 19
  2. 2024Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergencecited by 18
  3. 2021Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexitycited by 15
  4. 2024Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQRcited by 14
  5. 2023Fast Policy Learning for Linear Quadratic Control with Entropy Regularizationcited by 11
  6. 2023Revisiting LQR Control from the Perspective of Receding-Horizon Policy Gradientcited by 9
  7. 2021On the Sample Complexity of Decentralized Linear Quadratic Regulator with Partially Nested Information Structurecited by 7
  8. 2022Towards a Theoretical Foundation of Policy Optimization for Learning Control Policiescited by 6
  9. 2022Global Convergence of Direct Policy Search for State-Feedback H_∞ Robust Control: A Revisit of Nonsmooth Synthesis with Goldstein Subdifferentialcited by 6
  10. 2021Model-Free μ Synthesis via Adversarial Reinforcement Learningcited by 5
  11. 2022From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descentcited by 5
  12. 2023Convergence of Gradient-based MAML in LQRcited by 5
18 more
  1. 2021Asynchronous Distributed Reinforcement Learning for LQR Control via Zeroth-Order Block Coordinate Descentcited by 4
  2. 2023Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulatorscited by 4
  3. 2023Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithmscited by 4
  4. 2024Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial Statescited by 4
  5. 2021Learning Stochastic Optimal Policies via Gradient Descentcited by 2
  6. 2023Optimization Landscape of Policy Gradient Methods for Discrete-time Static Output Feedbackcited by 2
  7. 2024On the Complexity of Finite-Sum Smooth Optimization under the Polyak-Lojasiewicz Conditioncited by 2
  8. 2022On the Optimization Landscape of Dynamic Output Feedback: A Case Study for Linear Quadratic Regulatorcited by 1
  9. 2022Learning Decentralized Linear Quadratic Regulators with √T Regretcited by 1
  10. 2023Accelerated Optimization Landscape of Linear-Quadratic Regulatorcited by 0
  11. 2024Model-Free μ-Synthesis: A Nonsmooth Optimization Perspectivecited by 0
  12. 2024Distributed Policy Gradient for Linear Quadratic Networked Control with Limited Communication Rangecited by 0
  13. 2024Regret Analysis of Policy Optimization over Submanifolds for Linearly Constrained Online LQGcited by 0
  14. 2024Langevin Dynamics: A Unified Perspective on Optimization via Lyapunov Potentialscited by 0
  15. 2025Communication-aware Wide-Area Damping Control using Risk-Constrained Reinforcement Learningcited by 0
  16. 2026Scalar Federated Learning for Linear Quadratic Regulatorcited by 0
  17. 2026Global linear convergence of entropy-regularized softmax policy gradient beyond tabular MDPscited by 0
  18. 2026Sample Complexity of Policy Gradient for Log-Growth Controlcited by 0