mlsanityData to 7 Oct 2026

Regret Bounds for Risk-Sensitive Reinforcement Learning

arXiv 2210.05650 ↗First posted 11 Oct 2022cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

In safety-critical applications of reinforcement learning such as healthcare and robotics, it is often desirable to optimize risk-sensitive objectives that account for tail outcomes rather than expected reward. We prove the first regret bounds for reinforcement learning under a general class of risk-sensitive objectives including the popular CVaR objective. Our theory is based on a novel characterization of the CVaR objective as well as a novel optimistic MDP construction.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 11 of its 19 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

11 of 19 references.

References · 11

  1. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  2. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  3. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  4. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  5. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  6. 2014Optimizing the CVaR via Samplingcited by 98
  7. 2014Algorithms for CVaR Optimization in MDPscited by 98
  8. 2021Conservative Offline Distributional Reinforcement Learningcited by 78
  9. 2019Being Optimistic to Be Conservative: Quickly Learning a CVaR Policycited by 63
  10. 2019Worst Cases Policy Gradientscited by 52
  11. 2021Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 7; 2025: 7; 2026: 4.

26 papers.

Sort:

By area:

  1. 2023Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaRcited by 32
  2. 2024A Reductions Approach to Risk-Sensitive Reinforcement Learning with Optimized Certainty Equivalentscited by 15
  3. 2023Regret Bounds for Markov Decision Processes with Recursive Optimized Certainty Equivalentscited by 14
  4. 2023Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation and Human Feedbackcited by 13
  5. 2024RA-PbRL: Provably Efficient Risk-Aware Preference-Based Reinforcement Learningcited by 8
  6. 2023Regret Bounds for Risk-sensitive Reinforcement Learning with Lipschitz Dynamic Risk Measurescited by 7
  7. 2023Multi-objective Reinforcement Learning with Nonlinear Preferences: Provable Approximation for Maximizing Expected Scalarized Returncited by 7
  8. 2023Risk-sensitive Markov Decision Process and Learning under General Utility Functionscited by 7
  9. 2024Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Modelcited by 7
  10. 2023Provably Efficient CVaR RL in Low-rank MDPscited by 6
  11. 2025Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learningcited by 6
  12. 2024Spectral-Risk Safe Reinforcement Learning with Convergence Guaranteescited by 5
14 more
  1. 2024Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learningcited by 5
  2. 2025Near-Optimal Sample Complexity for Iterated CVaR Reinforcement Learning with a Generative Modelcited by 4
  3. 2024Provable Risk-Sensitive Distributional Reinforcement Learning with General Function Approximationcited by 3
  4. 2025Risk-aware Direct Preference Optimization under Nested Risk Measurecited by 2
  5. 2023Deep incremental learning models for financial temporal tabular datasets with distribution shiftscited by 1
  6. 2025Risk-Sensitive Agent Compositionscited by 1
  7. 2025Provably Efficient Sample Complexity for Robust CMDPcited by 1
  8. 2024Bellman Unbiasedness: Toward Provably Efficient Distributional Reinforcement Learning with General Value Function Approximationcited by 0
  9. 2025Reducing the Probability of Undesirable Outputs in Language Models Using Probabilistic Inferencecited by 0
  10. 2025Constrained Language Model Policy Optimization via Risk-aware Stepwise Alignmentcited by 0
  11. 2026Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinitycited by 0
  12. 2026Pessimistic Risk-Aware Policy Learning in Contextual Banditscited by 0
  13. 2026Theoretical Foundations of max@k Reinforcement Learningcited by 0
  14. 2026Continuity-Free Near-Minimax Leading-Order Regret for CVaR-UCBVIcited by 0