Cantelli Constrained Policy Optimization
On the map: Deep reinforcement learning methods → Safe and constrained RL
Abstract (arXiv)
We introduce Canary, a risk-averse method designed to optimize Value-at-Risk (VaR) constrained reinforcement learning (RL) problems. We employ Cantelli's inequality to obtain a tractable, conservative and smooth bound on the VaR constraint based on the first two moments of the cost return. This yields a constraint estimator that remains stable with tight violation thresholds in dense cost regimes. Extending the trust-region framework of the Constrained Policy Optimization (CPO) method, we further provide worst-case bounds for both policy improvement and constraint violation during the training process. Empirically during training, Canary is the only method that reliably satisfies the VaR constraint in every environment tested.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. First posted on arXiv 30 Jan 2026.
Builds on
0 of 0 references.
No references here yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Incentivizing Safer Actions in Policy Optimization for Constrained Reinforcement Learning
- 2024Off-Policy Primal-Dual Safe Reinforcement Learning
- 2025Risk-Averse Constrained Reinforcement Learning with Optimized Certainty Equivalents
- 2024Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence
- 2026Extreme Value Policy Optimization for Safe Reinforcement Learning
- 2024Constrained Reinforcement Learning Under Model Mismatch
- 2026Ratio-Variance Regularized Policy Optimization
- 2025Off-Policy Reinforcement Learning with Anytime Safety Guarantees via Robust Safe Gradient Flow