mlsanityData to 14 Oct 2026

Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence

arXiv 2406.03734 ↗First posted 6 Jun 2024math.OC cs.SY eess.SYcited by 1

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

In safety-critical applications, reinforcement learning (RL) needs to consider safety constraints. However, theoretical understandings of constrained RL for continuous control are largely absent. As a case study, this paper presents a cost-constrained LQR formulation, where a number of LQR costs with user-defined penalty matrices are subject to constraints. To solve it, we propose a policy gradient primal-dual method to find an optimal state feedback gain. Despite the non-convexity of the cost-constrained LQR problem, we provide a constructive proof for strong duality and a geometric interpretation of an optimal multiplier set. By proving that the concave dual function is Lipschitz smooth, we further provide convergence guarantees for the PG primal-dual method. Finally, we perform simulations to validate our theoretical findings.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 28 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

13 of 28 references.

References · 13

  1. 2019Introduction to Online Convex Optimizationcited by 803
  2. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
  3. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  4. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
  5. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 151
  6. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 57
  7. 2019Convergent Policy Optimization for Safe Reinforcement Learningcited by 57
  8. 2019Policy Optimization for H2 Linear Control with H_∞ Robustness Guarantee: Implicit Regularization and Global Convergencecited by 46
  9. 2019Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Gamescited by 42
  10. 2021Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRscited by 16
  11. 2024Data-Enabled Policy Optimization for Direct Adaptive Learning of the LQRcited by 9
  12. 2020Risk-Constrained Linear-Quadratic Regulatorscited by 8
1 more
  1. 2021Infinite-horizon Risk-constrained Linear Quadratic Regulator with Average Costcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Model-free LQG Control with Chance Constraintscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Off-Policy Primal-Dual Safe Reinforcement Learning
  2. 2024Adaptive Primal-Dual Method for Safe Reinforcement Learning
  3. 2024Foundations of Safe Online Reinforcement Learning in the Linear Quadratic Regulator: Generalized Baselines
  4. 2025Primal-dual policy learning for mean-field stochastic LQR problem ↗
  5. 2025Second-Order Policy Gradient Methods for the Linear Quadratic Regulator
  6. 2024Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning
  7. 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
  8. 2026Robust Peak-cost Constrained Reinforcement Learning