# Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRs

arXiv: https://arxiv.org/abs/2104.04901 · HTML: https://mlsanity.com/paper/2104.04901/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2021.

## Built on by (most cited first, 16 of 16)

- [Neural Operators of Backstepping Controller and Observer Gain Functions for Reaction-Diffusion PDEs](https://mlsanity.com/paper/2303.10506/) (2023)
- [Fast Policy Learning for Linear Quadratic Control with Entropy Regularization](https://mlsanity.com/paper/2311.14168/) (2023)
- [Towards a Theoretical Foundation of Policy Optimization for Learning Control Policies](https://mlsanity.com/paper/2210.04810/) (2022)
- [Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches](https://mlsanity.com/paper/2505.03706/) (2025)
- [Model-Free Learning for the Linear Quadratic Regulator over Rate-Limited Channels](https://mlsanity.com/paper/2401.01258/) (2024)
- [Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data](https://mlsanity.com/paper/2502.19977/) (2025)
- [Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence](https://mlsanity.com/paper/2406.03734/) (2024)
- [Asynchronous Parallel Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/2407.03233/) (2024)
- [On the Robustness of Derivative-free Methods for Linear Quadratic Regulator](https://mlsanity.com/paper/2506.12596/) (2025)
- [Model-Free Output Feedback Stabilization via Policy Gradient Methods](https://mlsanity.com/paper/2601.19284/) (2026)
- [Accelerated Optimization Landscape of Linear-Quadratic Regulator](https://mlsanity.com/paper/2307.03590/) (2023)
- [Policy Gradient-based Model Free Optimal LQG Control with a Probabilistic Risk Constraint](https://mlsanity.com/paper/2403.16767/) (2024)
- [Actor-Critic Learning for Risk-Constrained Linear Quadratic Regulation](https://mlsanity.com/paper/2510.22267/) (2025)
- [A Stochastic Gradient Descent Approach to Design Policy Gradient Methods for LQR](https://mlsanity.com/paper/2602.18933/) (2026)
- [Scalar Federated Learning for Linear Quadratic Regulator](https://mlsanity.com/paper/2604.05088/) (2026)
- [Model-free LQG Control with Chance Constraints](https://mlsanity.com/paper/2605.31310/) (2026)

## Abstract (arXiv)

While the techniques in optimal control theory are often model-based, the policy optimization (PO) approach directly optimizes the performance metric of interest. Even though it has been an essential approach for reinforcement learning problems, there is little theoretical understanding on its performance. In this paper, we focus on the risk-constrained linear quadratic regulator (RC-LQR) problem via the PO approach, which requires addressing a challenging non-convex constrained optimization problem. To solve it, we first build on our earlier result that an optimal policy has a time-invariant affine structure to show that the associated Lagrangian function is coercive, locally gradient dominated and has local Lipschitz continuous gradient, based on which we establish strong duality. Then, we design policy gradient primal-dual methods with global convergence guarantees in both model-based and sample-based settings. Finally, we use samples of system trajectories in simulations to validate our methods.