Data to 14 Oct 2026

Risk-Constrained Reinforcement Learning with Percentile Risk Criteria

arXiv 1512.01629 ↗First posted 5 Dec 2015cs.AI cs.LG math.OCcited by 337

Not on the map. Why?

Cited by 118 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In many sequential decision-making problems one is interested in minimizing an expected cumulative cost while taking into account risk, i.e., increased awareness of events of small probability and high consequences. Accordingly, the objective of this paper is to present efficient reinforcement learning algorithms for risk-constrained Markov decision processes (MDPs), where risk is represented via a chance constraint or a constraint on the conditional value-at-risk (CVaR) of the cumulative cost. We collectively refer to such problems as percentile risk-constrained MDPs. Specifically, we first derive a formula for computing the gradient of the Lagrangian function for percentile risk-constrained MDPs. Then, we devise policy gradient and actor-critic algorithms that (1) estimate such gradient, (2) update the policy in the descent direction, and (3) update the Lagrange multiplier in the ascent direction. For these algorithms we prove convergence to locally optimal policies. Finally, we demonstrate the effectiveness of our algorithms in an optimal stopping problem and an online marketing application.

Key facts. Cited by 337 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 Dec 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 50; 2022: 58; 2023: 60; 2024: 65; 2025: 51; 2026: 52.

337 papers.

Sort:

By area:

2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
2021Recent Advances in Reinforcement Learning in Financecited by 110
2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 91
2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 80
2021Conservative Offline Distributional Reinforcement Learningcited by 79
2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 71
2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 69
2022Penalized Proximal Policy Optimization for Safe Reinforcement Learningcited by 68

38 more

2022Probable Domain Generalization via Quantile Risk Minimizationcited by 66
2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Modelcited by 64
2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 60
2021Safe Reinforcement Learning Using Robust Control Barrier Functionscited by 59
2022Reachability Constrained Reinforcement Learningcited by 59
2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 59
2022Constrained Policy Optimization via Bayesian World Modelscited by 57
2024Enhancing LLM Safety via Constrained Direct Preference Optimizationcited by 49
2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 47
2023Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotioncited by 47
2022Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Riskcited by 41
2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
2022Uncertainty Estimation for Language Reward Modelscited by 37
2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 37
2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 36
2021Multi-Agent Constrained Policy Optimisationcited by 35
2021DOPE: Doubly Optimistic and Pessimistic Exploration for Safe Reinforcement Learningcited by 35
2021A Dual Approach to Constrained Markov Decision Processes with Entropy Regularizationcited by 34
2021Model-based Constrained Reinforcement Learning using Generalized Control Barrier Functioncited by 32
2021Feasible Actor-Critic: Constrained Reinforcement Learning for Ensuring Statewise Safetycited by 30
2022Challenging Common Assumptions in Convex Reinforcement Learningcited by 30
2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 29
2021Off-Policy Risk Assessment in Contextual Banditscited by 29
2023Safe Offline Reinforcement Learning with Real-Time Budget Constraintscited by 28
2023Last-Iterate Convergent Policy Gradient Primal-Dual Methods for Constrained MDPscited by 28
2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 28
2021Joint Synthesis of Safety Certificate and Safe Control Policy using Constrained Reinforcement Learningcited by 27
2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 27
2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
2022Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPscited by 27
2021Safe Pontryagin Differentiable Programmingcited by 26
2022Safe Reinforcement Learning Using Black-Box Reachability Analysiscited by 26
2022Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 26
2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 23
2023Causal Reinforcement Learning: A Surveycited by 23
2024Truly No-Regret Learning in Constrained MDPscited by 23
2024CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learningcited by 23
2022Provably Efficient Risk-Sensitive Reinforcement Learning: Iterated CVaR and Worst Pathcited by 22

50 most cited of 337 All 337 (JSON)