mlsanityData to 7 Oct 2026

Risk-Aware General-Utility Markov Decision Processes

arXiv 2607.09298 ↗First posted 10 Jul 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We study general-utility Markov decision processes (GUMDPs) with risk-aware objectives. In this framework, an agent aims to optimize a risk measure of the distribution of objective values, where the objective function depends on the frequency of visitation of states induced by the agent's policy. First, we motivate, propose, and formalize risk-aware GUMDPs, which enable agents and decision makers to trade off expected performance by risk aversion while benefiting from the rich set of objectives that can be cast under the framework of GUMDPs. We focus our attention on the entropic risk measure (ERM). Second, we show how we can solve risk-aware GUMDPs with ERM objectives by resorting to online planning techniques. In particular, we propose an approach based on Monte Carlo Tree Search (MCTS) to provably solve risk-aware GUMDPs up to any desired accuracy. Third, we provide a set of experimental results showcasing that our approach is successful when optimizing for a spectrum of risk-aware behaviors in the context of GUMDPs under diverse tasks (standard MDPs, maximum state entropy exploration, imitation learning, and multi-objective MDPs).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 42 references that are in the map. First posted on arXiv 10 Jul 2026.

Builds on

17 of 42 references.

References · 17

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  4. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  5. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  6. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  7. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 162
  8. 2020Exploration-Exploitation in Constrained MDPscited by 146
  9. 2018Variational Option Discovery Algorithmscited by 121
  10. 2018Universal Successor Features Approximatorscited by 102
  11. 2021Reward is enough for convex MDPscited by 82
  12. 2021Concave Utility Reinforcement Learning: the Mean-Field Game Viewpointcited by 54
5 more
  1. 2019Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Surveycited by 45
  2. 2020Cautious Reinforcement Learning via Distributional Risk in the Dual Domaincited by 18
  3. 2025Efficient Risk-sensitive Planning via Entropic Risk Measurescited by 4
  4. 2025Recursive Entropic Risk Optimization in Discounted MDPs: Sample Complexity Bounds with a Generative Modelcited by 3
  5. 2025Solving General-Utility Markov Decision Processes in the Single-Trial Regime with Online Planningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning
  2. 2024Learning Utilities from Demonstrations in Markov Decision Processes
  3. 2021Robust Entropy-regularized Markov Decision Processes
  4. 2021An Offline Risk-aware Policy Selection Method for Bayesian Markov Decision Processes
  5. 2025Generalization in Monitored Markov Decision Processes (Mon-MDPs)
  6. 2026Towards Optimal Policy Improvement
  7. 2025Risk-sensitive Actor-Critic with Static Spectral Risk Measures for Online and Offline Reinforcement Learning
  8. 2023Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization