mlsanityData to 14 Oct 2026

Policy Gradient-based Model Free Optimal LQG Control with a Probabilistic Risk Constraint

arXiv 2403.16767 ↗First posted 25 Mar 2024eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

In this paper, we investigate a model-free optimal control design that minimizes an infinite horizon average expected quadratic cost of states and control actions subject to a probabilistic risk or chance constraint using input-output data. In particular, we consider linear time-invariant systems and design an optimal controller within the class of linear state feedback control. Three different policy gradient (PG) based algorithms, natural policy gradient (NPG), Gauss-Newton policy gradient (GNPG), and deep deterministic policy gradient (DDPG), are developed, and compared with the optimal risk-neutral linear-quadratic regulator (LQR) and a scenario-based model predictive control (MPC) technique via numerical simulations. The convergence properties and the accuracy of all the algorithms are compared numerically. We also establish analytical convergence properties of the NPG and GNPG algorithms under the known model scenario, while the proof of convergence for the unknown model scenario is part of our ongoing work.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 27 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

13 of 27 references.

References · 13

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  4. 2017Towards Generalization and Simplicity in Continuous Controlcited by 68
  5. 2013The Scenario Approach for Stochastic Model Predictive Control with Bounds on Closed-Loop Constraint Violationscited by 24
  6. 2021Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRscited by 16
  7. 2020Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guaranteecited by 9
  8. 2021Efficient Off-Policy Q-Learning for Data-Based Discrete-Time LQR Problemscited by 9
  9. 2020Risk-Constrained Linear-Quadratic Regulatorscited by 8
  10. 2020Primal-dual Learning for the Model-free Risk-constrained Linear Quadratic Regulatorcited by 8
  11. 2021Infinite-horizon Risk-constrained Linear Quadratic Regulator with Average Costcited by 7
  12. 2022Stochastic MPC with robustness to bounded parametric uncertaintycited by 3
1 more
  1. 2021Data-driven tube-based stochastic predictive controlcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Model-free LQG Control with Chance Constraints
  2. 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
  3. 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression
  4. 2025On the Gradient Domination of the LQG Problem
  5. 2025Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches
  6. 2026(Cheap) Stochastic Policy Gradient Converges with High Probability for Linear Quadratic Regulator
  7. 2026A Stochastic Gradient Descent Approach to Design Policy Gradient Methods for LQR
  8. 2025Policy Gradient Method for LQG Control via Input-Output-History Representation: Convergence to O(ε)-Stationary Points