mlsanityData to 7 Oct 2026

Optimizing the CVaR via Sampling

arXiv 1404.3862 ↗First posted 15 Apr 2014stat.ML cs.AI cs.LGcited by 98

Not on the map. Why?

Abstract (arXiv)

Conditional Value at Risk (CVaR) is a prominent risk measure that is being used extensively in various domains. We develop a new formula for the gradient of the CVaR in the form of a conditional expectation. Based on this formula, we propose a novel sampling-based estimator for the CVaR gradient, in the spirit of the likelihood-ratio method. We analyze the bias of the estimator, and prove the convergence of a corresponding stochastic gradient descent algorithm to a local CVaR optimum. Our method allows to consider CVaR optimization in new domains. As an example, we consider a reinforcement learning application, and learn a risk-sensitive controller for the game of Tetris.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Apr 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 2; 2021: 20; 2022: 18; 2023: 11; 2024: 10; 2025: 14; 2026: 21.

98 papers.

Sort:

By area:

  1. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  2. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  3. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  4. 2022A Survey on Model-based Reinforcement Learningcited by 65
  5. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 63
  6. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 46
  7. 2022Efficient Risk-Averse Reinforcement Learningcited by 41
  8. 2022Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Riskcited by 40
  9. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  10. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  11. 2023Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaRcited by 32
  12. 2020Bayesian Robust Optimization for Imitation Learningcited by 28
38 more
  1. 2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 28
  2. 2019Risk-Aware Active Inverse Reinforcement Learningcited by 26
  3. 2022Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 26
  4. 2022Provably Efficient Risk-Sensitive Reinforcement Learning: Iterated CVaR and Worst Pathcited by 21
  5. 2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 21
  6. 2021On the Convergence and Optimality of Policy Gradient for Markov Coherent Riskcited by 19
  7. 2021Unsupervised Reinforcement Learning in Multiple Environmentscited by 17
  8. 2018Risk-Sensitive Generative Adversarial Imitation Learningcited by 16
  9. 2022Robust Policy Learning over Multiple Uncertainty Setscited by 16
  10. 2021SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learningcited by 15
  11. 2024A Reductions Approach to Risk-Sensitive Reinforcement Learning with Optimized Certainty Equivalentscited by 15
  12. 2022Adversarial Unlearning: Reducing Confidence Along Adversarial Directionscited by 14
  13. 2021Lyapunov Barrier Policy Optimizationcited by 12
  14. 2021Learning Risk-aware Costmaps for Traversability in Challenging Environmentscited by 12
  15. 2023An Alternative to Variance: Gini Deviation for Risk-averse Policy Gradientcited by 12
  16. 2025RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Trainingcited by 12
  17. 2021Policy Gradient Bayesian Robust Optimization for Imitation Learningcited by 11
  18. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 11
  19. 2021Variance Penalized On-Policy and Off-Policy Actor-Criticcited by 10
  20. 2023ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantagescited by 10
  21. 2022Distributional Hamilton-Jacobi-Bellman Equations for Continuous-Time Reinforcement Learningcited by 9
  22. 2021Spectral risk-based learning using unbounded lossescited by 8
  23. 2021Two steps to risk sensitivitycited by 8
  24. 2023Quasi-optimal Reinforcement Learning with Continuous Actionscited by 7
  25. 2023Risk-Aware Reinforcement Learning through Optimal Transport Theorycited by 7
  26. 2024A Simple Mixture Policy Parameterization for Improving Sample Efficiency of CVaR Optimizationcited by 7
  27. 2021Robust Model-based Reinforcement Learning for Autonomous Greenhouse Controlcited by 6
  28. 2022Risk-Sensitive Policy with Distributional Reinforcement Learningcited by 6
  29. 2023Provably Efficient CVaR RL in Low-rank MDPscited by 6
  30. 2026Efficient Tail-Aware Generative Optimization via Flow Model Fine-Tuningcited by 6
  31. 2021Risk-Aware Learning for Scalable Voltage Optimization in Distribution Gridscited by 5
  32. 2021Planning for Risk-Aversion and Expected Value in MDPscited by 5
  33. 2022Learning to Optimize with Stochastic Dominance Constraintscited by 5
  34. 2023Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theorycited by 5
  35. 2025Return Capping: Sample-Efficient CVaR Policy Gradient Optimisationcited by 5
  36. 2025ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learningcited by 5
  37. 2022Non-stationary Risk-sensitive Reinforcement Learning: Near-optimal Dynamic Regret, Adaptive Detection, and Separation Designcited by 4
  38. 2023Risk-Averse Reinforcement Learning via Dynamic Time-Consistent Risk Measurescited by 4

50 most cited of 98 All 98 (JSON)