mlsanityData to 7 Oct 2026

Markov Decision Processes with Recursive Risk Measures

arXiv 2010.07220 ↗First posted 14 Oct 2020math.OC q-fin.RMcited by 17

Not on the map. Why?

Abstract (arXiv)

In this paper, we consider risk-sensitive Markov Decision Processes (MDPs) with Borel state and action spaces and unbounded cost under both finite and infinite planning horizons. Our optimality criterion is based on the recursive application of static risk measures. This is motivated by recursive utilities in the economic literature, has been studied before for the entropic risk measure and is extended here to an axiomatic characterization of suitable risk measures. We derive a Bellman equation and prove the existence of Markovian optimal policies. For an infinite planning horizon, the model is shown to be contractive and the optimal policy to be stationary. Moreover, we establish a connection to distributionally robust MDPs, which provides a global interpretation of the recursively defined objective function. Monotone models are studied in particular.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 2; 2023: 4; 2024: 2; 2025: 5; 2026: 2.

17 papers.

Sort:

  1. 2022Provably Efficient Risk-Sensitive Reinforcement Learning: Iterated CVaR and Worst Pathcited by 21
  2. 2021Reinforcement Learning with Dynamic Convex Risk Measurescited by 20
  3. 2023Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexitycited by 16
  4. 2023Regret Bounds for Markov Decision Processes with Recursive Optimized Certainty Equivalentscited by 14
  5. 2021Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal controlcited by 12
  6. 2022Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learningcited by 9
  7. 2023Regret Bounds for Risk-sensitive Reinforcement Learning with Lipschitz Dynamic Risk Measurescited by 7
  8. 2024Risk-averse Total-reward MDPs with ERM and EVaRcited by 4
  9. 2025Efficient Risk-sensitive Planning via Entropic Risk Measurescited by 4
  10. 2025Near-Optimal Sample Complexity for Iterated CVaR Reinforcement Learning with a Generative Modelcited by 4
  11. 2023Is Risk-Sensitive Reinforcement Learning Properly Resolved?cited by 2
  12. 2025Risk-sensitive Reinforcement Learning Based on Convex Scoring Functionscited by 2
5 more
  1. 2024Robust Reinforcement Learning with Dynamic Distortion Risk Measurescited by 1
  2. 2026Reinforcement Learning with Markov Risk Measures and Multipattern Risk Approximationcited by 1
  3. 2025Risk-Averse Total-Reward Reinforcement Learningcited by 0
  4. 2025Neural Architecture Search for global multi-step Forecasting of Energy Production Time Seriescited by 0
  5. 2026On the Sample Complexity of Discounted Reinforcement Learning with Optimized Certainty Equivalentscited by 0