mlsanityData to 7 Oct 2026

Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach

arXiv 1506.02188 ↗First posted 6 Jun 2015cs.AI math.OCcited by 162

Not on the map. Why?

Abstract (arXiv)

In this paper we address the problem of decision making within a Markov decision process (MDP) framework where risk and modeling errors are taken into account. Our approach is to minimize a risk-sensitive conditional-value-at-risk (CVaR) objective, as opposed to a standard risk-neutral expectation. We refer to such problem as CVaR MDP. Our first contribution is to show that a CVaR objective, besides capturing risk sensitivity, has an alternative interpretation as expected cost under worst-case modeling errors, for a given error budget. This result, which is of independent interest, motivates CVaR MDPs as a unifying framework for risk-sensitive and robust decision making. Our second contribution is to present an approximate value-iteration algorithm for CVaR MDPs and analyze its convergence rate. To our knowledge, this is the first solution algorithm for CVaR MDPs that enjoys error guarantees. Finally, we present results from numerical experiments that corroborate our theoretical findings and show the practicality of our approach.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 2; 2021: 28; 2022: 25; 2023: 26; 2024: 20; 2025: 30; 2026: 30.

162 papers.

Sort:

By area:

  1. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  2. 2021Recent Advances in Reinforcement Learning in Financecited by 102
  3. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  4. 2022COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimationcited by 69
  5. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  6. 2022Efficient Risk-Averse Reinforcement Learningcited by 41
  7. 2022Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Riskcited by 40
  8. 2021State Augmented Constrained Reinforcement Learning: Overcoming the Limitations of Learning with Rewardscited by 37
  9. 2022Probabilistically Robust Learning: Balancing Average- and Worst-case Performancecited by 36
  10. 2023Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaRcited by 32
  11. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  12. 2022Challenging Common Assumptions in Convex Reinforcement Learningcited by 29
38 more
  1. 2020Bayesian Robust Optimization for Imitation Learningcited by 28
  2. 2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 28
  3. 2021STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigationcited by 27
  4. 2019Risk-Aware Active Inverse Reinforcement Learningcited by 26
  5. 2022Provably Efficient Risk-Sensitive Reinforcement Learning: Iterated CVaR and Worst Pathcited by 21
  6. 2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 21
  7. 2021On the Convergence and Optimality of Policy Gradient for Markov Coherent Riskcited by 19
  8. 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 19
  9. 2021Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimizationcited by 18
  10. 2023On Dynamic Programming Decompositions of Static Risk Measures in Markov Decision Processescited by 17
  11. 2021SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learningcited by 15
  12. 2022Distributionally Robust Optimization via Ball Oracle Accelerationcited by 14
  13. 2021Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigationcited by 13
  14. 2022Deep Hedging: Continuous Reinforcement Learning for Hedging of General Portfolios across Multiple Risk Aversionscited by 13
  15. 2023Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation and Human Feedbackcited by 13
  16. 2021Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal controlcited by 12
  17. 2022RAP: Risk-Aware Prediction for Robust Planningcited by 12
  18. 2023Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learningcited by 12
  19. 2023Robust Fitted-Q-Evaluation and Iteration under Sequentially Exogenous Unobserved Confounderscited by 12
  20. 2023An Alternative to Variance: Gini Deviation for Risk-averse Policy Gradientcited by 12
  21. 2024Constrained Reinforcement Learning with Smoothed Log Barrier Functioncited by 12
  22. 2025A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safetycited by 12
  23. 2025RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Trainingcited by 12
  24. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 11
  25. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
  26. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 11
  27. 2021Safe Distributional Reinforcement Learningcited by 10
  28. 2023Distributional Model Equivalence for Risk-Sensitive Reinforcement Learningcited by 9
  29. 2021Risk-Averse Stochastic Shortest Path Planningcited by 8
  30. 2021Robust Value Iteration for Continuous Control Taskscited by 8
  31. 2021Two steps to risk sensitivitycited by 8
  32. 2022Game-Theoretic Planning for Autonomous Driving among Risk-Aware Human Driverscited by 7
  33. 2023Regret Bounds for Risk-sensitive Reinforcement Learning with Lipschitz Dynamic Risk Measurescited by 7
  34. 2024A Simple Mixture Policy Parameterization for Improving Sample Efficiency of CVaR Optimizationcited by 7
  35. 2025Risk-Averse Finetuning of Large Language Modelscited by 7
  36. 2021Continuous-Time Fitted Value Iteration for Robust Policiescited by 6
  37. 2022SAAC: Safe Reinforcement Learning as an Adversarial Game of Actor-Criticscited by 6
  38. 2022Risk Verification of Stochastic Systems with Neural Network Controllerscited by 6

50 most cited of 162 All 162 (JSON)