mlsanityData to 14 Oct 2026

Regret Bounds for Risk-sensitive Reinforcement Learning with Lipschitz Dynamic Risk Measures

arXiv 2306.02399 ↗First posted 4 Jun 2023cs.LG cs.AI stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

We study finite episodic Markov decision processes incorporating dynamic risk measures to capture risk sensitivity. To this end, we present two model-based algorithms applied to Lipschitz dynamic risk measures, a wide range of risk measures that subsumes spectral risk measure, optimized certainty equivalent, distortion risk measures among others. We establish both regret upper bounds and lower bounds. Notably, our upper bounds demonstrate optimal dependencies on the number of actions and episodes, while reflecting the inherent trade-off between risk sensitivity and sample complexity. Additionally, we substantiate our theoretical results through numerical experiments.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 21 of its 68 references that are in the map. First posted on arXiv 4 Jun 2023.

Builds on

21 of 68 references.

References · 21

  1. 2017Minimax Regret Bounds for Reinforcement Learningcited by 503
  2. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 234
  3. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 170
  4. 2020Episodic Reinforcement Learning in Finite MDPs: Minimax Lower Bounds Revisitedcited by 106
  5. 2019Non-Asymptotic Gap-Dependent Regret Bounds for Tabular MDPscited by 105
  6. 2016Explore First, Exploit Next: The True Shape of Regret in Bandit Problemscited by 79
  7. 2020Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regretcited by 58
  8. 2021Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 43
  9. 2023Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaRcited by 32
  10. 2021Off-Policy Risk Assessment in Contextual Banditscited by 29
  11. 2022Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 26
  12. 2022Provably Efficient Risk-Sensitive Reinforcement Learning: Iterated CVaR and Worst Pathcited by 22
9 more
  1. 2021Reinforcement Learning with Dynamic Convex Risk Measurescited by 21
  2. 2019A Wasserstein distance approach for concentration of empirical risk estimatescited by 18
  3. 2020Markov Decision Processes with Recursive Risk Measurescited by 18
  4. 2022Bridging Distributional and Risk-sensitive Reinforcement Learning with Provable Regret Boundscited by 16
  5. 2023Regret Bounds for Markov Decision Processes with Recursive Optimized Certainty Equivalentscited by 15
  6. 2020Minimizing Spectral Risk Measures Applied to Markov Decision Processescited by 12
  7. 2022Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learningcited by 9
  8. 2022Non-stationary Risk-sensitive Reinforcement Learning: Near-optimal Dynamic Regret, Adaptive Detection, and Separation Designcited by 4
  9. 2016Practicality of Nested Risk Measures for Dynamic Electric Vehicle Chargingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 1; 2026: 3.

8 papers.

  1. 2025Recursive Entropic Risk Optimization in Discounted MDPs: Sample Complexity Bounds with a Generative Modelcited by 5
  2. 2024Provable Risk-Sensitive Distributional Reinforcement Learning with General Function Approximationcited by 3
  3. 2023A Distribution Optimization Framework for Confidence Bounds of Risk Measurescited by 2
  4. 2024On the Approximation and Convergence of Distributional Policy Gradient Algorithms for Risk-Sensitive Reinforcement Learningcited by 1
  5. 2026On the Sample Complexity of Discounted Reinforcement Learning with Optimized Certainty Equivalentscited by 1
  6. 2024Provably Efficient Partially Observable Risk-Sensitive Reinforcement Learning with Hindsight Observationcited by 0
  7. 2026Continuity-Free Near-Minimax Leading-Order Regret for CVaR-UCBVIcited by 0
  8. 2026Near-Optimal Sample Complexity for Recursive Entropic Risk Reinforcement Learning with a Generative Modelcited by 0