mlsanityData to 14 Oct 2026

Actor-Critic Learning for Risk-Constrained Linear Quadratic Regulation

arXiv 2510.22267 ↗First posted 25 Oct 2025math.OCcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

In this paper, we investigate the infinite-horizon risk-constrained linear quadratic regulator problem (RC-QR), which augments the classical LQR formulation with a statistical constraint on the variability of the system state to incorporate risk awareness, a key requirement in safety-critical control applications. We propose an actor-critic learning algorithm that jointly performs policy evaluation and policy improvement in a model-free and online manner. The RC-QR problem is first reformulated as a max-min optimization problem, from which we develop a multi-time-scale stochastic approximation scheme. The critic employs temporal-difference learning to estimate the action-value function, the actor updates the policy parameters via a policy gradient step, and the dual variable is adapted through gradient ascent to enforce the risk constraint.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 37 references that are in the map. First posted on arXiv 25 Oct 2025.

Builds on

18 of 37 references.

References · 18

  1. 2015Trust Region Policy Optimizationcited by 2,580
  2. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  3. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 194
  4. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  5. 2020Optimizing Static Linear Feedback: Gradient Methodcited by 27
  6. 2021A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learningcited by 24
  7. 2018Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systemscited by 23
  8. 2019Actor-Critic Provably Finds Nash Equilibria of Linear-Quadratic Mean-Field Gamescited by 19
  9. 2021Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRscited by 16
  10. 2019A Risk-Sensitive Finite-Time Reachability Approach for Safety of Stochastic Dynamic Systemscited by 15
  11. 2021Risk-sensitive safety analysis using Conditional Value-at-Riskcited by 14
  12. 2022On the Optimization Landscape of Dynamic Output Feedback Linear Quadratic Controlcited by 11
6 more
  1. 2021Separation of Learning and Control for Cyber-Physical Systemscited by 10
  2. 2020Risk-Constrained Linear-Quadratic Regulatorscited by 8
  3. 2021Infinite-horizon Risk-constrained Linear Quadratic Regulator with Average Costcited by 7
  4. 2022Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Controlcited by 7
  5. 2024Risk-Aware Finite-Horizon Social Optimal Control of Mean-Field Coupled Linear-Quadratic Subsystemscited by 1
  6. 2025Combined Learning and Control: A New Paradigm for Optimal Control with Unknown Dynamicscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Foundations of Safe Online Reinforcement Learning in the Linear Quadratic Regulator: Generalized Baselines
  2. 2026Model-free LQG Control with Chance Constraints
  3. 2026Data-Driven Continuous-Time Linear Quadratic Regulator via Closed-Loop and Reinforcement Learning Parameterizations
  4. 2026A Stochastic Gradient Descent Approach to Design Policy Gradient Methods for LQR
  5. 2024Stability-Certified On-Policy Data-Driven LQR via Recursive Learning and Policy Gradient
  6. 2024Policy Gradient-based Model Free Optimal LQG Control with a Probabilistic Risk Constraint
  7. 2024Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence
  8. 2025Ergodic-Risk Constrained Policy Optimization: The Linear Quadratic Case ↗