mlsanityData to 14 Oct 2026

Global Convergence of Policy Gradient Methods for ReLU Controllers in Linear Quadratic Regulation

arXiv 2604.22138 ↗First posted 24 Apr 2026math.OC cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

We study the convergence of model-based policy gradient for the deterministic, scalar, discounted linear-quadratic regulator when the controller is an overparameterized one-hidden-layer ReLU network without biases. Although the optimal LQR controller is linear, neural parameterization creates a redundant nonconvex weight space with a possibly asymmetric piecewise-linear controller. We show that this structure can still be analyzed exactly through the two effective gains induced on the positive and negative half-lines. Under suitable random initialization, sufficient width, and a small step size, the model-based policy gradient remains stable, decreases the cost geometrically, and drives the effective gains to the unique optimal scalar LQR gain with high probability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 17 references that are in the map. First posted on arXiv 24 Apr 2026.

Builds on

13 of 17 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Sample Complexity of the Linear Quadratic Regulator: A Reinforcement Learning Lens16 Apr

Other references · 12

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  3. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  4. 2018On the Optimization of Deep Networks: Implicit Acceleration by Overparameterizationcited by 284
  5. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  6. 2018Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balancedcited by 177
  7. 2019Neural Policy Gradient Methods: Global Optimality and Rates of Convergencecited by 159
  8. 2019Global Optimality Guarantees For Policy Gradient Methodscited by 150
  9. 2018Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systemscited by 67
  10. 2022Neural System Level Synthesis: Learning over All Stabilizing Policies for Nonlinear Systemscited by 11
  11. 2024Convergence Analysis of Gradient Flow for Overparameterized LQR Formulationscited by 3
  12. 2025On the (almost) Global Exponential Convergence of the Overparameterized Policy Optimization for the LQR Problemcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Second-Order Policy Gradient Methods for the Linear Quadratic Regulator
  2. 2025On the Gradient Domination of the LQG Problem
  3. 2026(Cheap) Stochastic Policy Gradient Converges with High Probability for Linear Quadratic Regulator
  4. 2026Gradient Dominance in the Linear Quadratic Regulator: A Unified Analysis for Continuous-Time and Discrete-Time Systems
  5. 2024Stability-Certified On-Policy Data-Driven LQR via Recursive Learning and Policy Gradient
  6. 2025Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches
  7. 2025Convergence of Flow-Policy Gradient Learning for Linear Quadratic Regulator Problems
  8. 2024Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQR