# Policy Gradient-based Model Free Optimal LQG Control with a Probabilistic Risk Constraint

arXiv: https://arxiv.org/abs/2403.16767 · HTML: https://mlsanity.com/paper/2403.16767/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 13 of 13)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Towards Generalization and Simplicity in Continuous Control](https://mlsanity.com/paper/1703.02660/) (2017)
- [The Scenario Approach for Stochastic Model Predictive Control with Bounds on Closed-Loop Constraint Violations](https://mlsanity.com/paper/1307.5640/) (2013)
- [Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRs](https://mlsanity.com/paper/2104.04901/) (2021)
- [Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee](https://mlsanity.com/paper/2011.06882/) (2020)
- [Efficient Off-Policy Q-Learning for Data-Based Discrete-Time LQR Problems](https://mlsanity.com/paper/2105.07761/) (2021)
- [Risk-Constrained Linear-Quadratic Regulators](https://mlsanity.com/paper/2004.04685/) (2020)
- [Primal-dual Learning for the Model-free Risk-constrained Linear Quadratic Regulator](https://mlsanity.com/paper/2011.10931/) (2020)
- [Infinite-horizon Risk-constrained Linear Quadratic Regulator with Average Cost](https://mlsanity.com/paper/2103.15363/) (2021)
- [Stochastic MPC with robustness to bounded parametric uncertainty](https://mlsanity.com/paper/2205.10275/) (2022)
- [Data-driven tube-based stochastic predictive control](https://mlsanity.com/paper/2112.04439/) (2021)

## Abstract (arXiv)

In this paper, we investigate a model-free optimal control design that minimizes an infinite horizon average expected quadratic cost of states and control actions subject to a probabilistic risk or chance constraint using input-output data. In particular, we consider linear time-invariant systems and design an optimal controller within the class of linear state feedback control. Three different policy gradient (PG) based algorithms, natural policy gradient (NPG), Gauss-Newton policy gradient (GNPG), and deep deterministic policy gradient (DDPG), are developed, and compared with the optimal risk-neutral linear-quadratic regulator (LQR) and a scenario-based model predictive control (MPC) technique via numerical simulations. The convergence properties and the accuracy of all the algorithms are compared numerically. We also establish analytical convergence properties of the NPG and GNPG algorithms under the known model scenario, while the proof of convergence for the unknown model scenario is part of our ongoing work.