mlsanityData to 14 Oct 2026

Model-free LQG Control with Chance Constraints

arXiv 2605.31310 ↗First posted 29 May 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

This paper studies model-free optimal control design and its convergence properties for linear time-invariant systems subject to probabilistic risk or chance constraints. In particular, we study a natural policy gradient (NPG)-based actor-critic (AC) algorithm with two timescales, using a Lagrangian primal-dual framework to enforce the constraint. Furthermore, the risk is defined as the probability that a function of the one-step-ahead state exceeds a user-specified threshold. To our knowledge, this is the first work to study the analytical convergence properties for NPG-based AC in a chance-constrained linear-quadratic Gaussian (LQG) regulator setting without model knowledge. We establish the coercivity and gradient dominance properties of the Lagrangian function, which ensure linear convergence and closed-loop stability during training for the actor. On the other hand, we analyse the convergence properties of the temporal difference (TD(0)) learning for the critic, applying stochastic approximation theory. Also, we demonstrate no duality gap in the constrained optimisation problem. Additionally, we have performed numerical analysis of the convergence properties and accuracy of the proposed method, comparing it with model-based chance-constrained LQR and scenario-based MPC. Results show that our approach effectively limits risk while maintaining near-optimal performance, without requiring full model knowledge or real-time optimisation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 26 of its 46 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

26 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Gradient Dominance in the Linear Quadratic Regulator: A Unified Analysis for Continuous-Time and Discrete-Time Systems26 Feb
  2. 2024Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence6 Jun

Other references · 24

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2015An Introduction to Matrix Concentration Inequalitiescited by 397
  4. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  5. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  6. 2020A Finite Time Analysis of Two Time-Scale Actor Critic Methodscited by 122
  7. 2017Towards Generalization and Simplicity in Continuous Controlcited by 68
  8. 2019Policy Optimization for H2 Linear Control with H_∞ Robustness Guarantee: Implicit Regularization and Global Convergencecited by 46
  9. 2021Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Controlcited by 28
  10. 2013The Scenario Approach for Stochastic Model Predictive Control with Bounds on Closed-Loop Constraint Violationscited by 24
  11. 2025CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functionscited by 17
  12. 2021Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRscited by 16
12 more
  1. 2024A Simple Finite-Time Analysis of TD Learning with Linear Function Approximationcited by 16
  2. 2022Single Time-scale Actor-critic Method to Solve the Linear Quadratic Regulator with Convergence Guaranteescited by 14
  3. 2020Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guaranteecited by 9
  4. 2021Efficient Off-Policy Q-Learning for Data-Based Discrete-Time LQR Problemscited by 9
  5. 2020Risk-Constrained Linear-Quadratic Regulatorscited by 8
  6. 2024Learning Control Barrier Functions and their application in Reinforcement Learning: A Surveycited by 8
  7. 2021Infinite-horizon Risk-constrained Linear Quadratic Regulator with Average Costcited by 7
  8. 2022Stochastic MPC with robustness to bounded parametric uncertaintycited by 3
  9. 2025Revisiting Strong Duality, Hidden Convexity, and Gradient Dominance in the Linear Quadratic Regulatorcited by 3
  10. 2021Data-driven tube-based stochastic predictive controlcited by 2
  11. 2025Global Optimality of Single-Timescale Actor-Critic under Continuous State-Action Space: A Study on Linear Quadratic Regulatorcited by 2
  12. 2014Linear Controller Design for Chance Constrained Systemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Policy Gradient-based Model Free Optimal LQG Control with a Probabilistic Risk Constraint
  2. 2025On the Gradient Domination of the LQG Problem
  3. 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
  4. 2026(Cheap) Stochastic Policy Gradient Converges with High Probability for Linear Quadratic Regulator
  5. 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression
  6. 2025Actor-Critic Learning for Risk-Constrained Linear Quadratic Regulation
  7. 2025Primal-dual policy learning for mean-field stochastic LQR problem ↗
  8. 2025Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming