# Minimizing Spectral Risk Measures Applied to Markov Decision Processes

arXiv: https://arxiv.org/abs/2012.04521 · HTML: https://mlsanity.com/paper/2012.04521/

## Key facts

- Cited by 12 papers in the map (data through 13 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 12 of 12)

- [Reinforcement Learning with Dynamic Convex Risk Measures](https://mlsanity.com/paper/2112.13414/) (2021)
- [A Reductions Approach to Risk-Sensitive Reinforcement Learning with Optimized Certainty Equivalents](https://mlsanity.com/paper/2403.06323/) (2024)
- [Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control](https://mlsanity.com/paper/2109.08947/) (2021)
- [Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning](https://mlsanity.com/paper/2206.14666/) (2022)
- [Regret Bounds for Risk-sensitive Reinforcement Learning with Lipschitz Dynamic Risk Measures](https://mlsanity.com/paper/2306.02399/) (2023)
- [Supervised Learning with General Risk Functionals](https://mlsanity.com/paper/2206.13648/) (2022)
- [Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learning](https://mlsanity.com/paper/2501.02087/) (2025)
- [Spectral-Risk Safe Reinforcement Learning with Convergence Guarantees](https://mlsanity.com/paper/2405.18698/) (2024)
- [Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity](https://mlsanity.com/paper/2208.03066/) (2022)
- [Risk-sensitive Actor-Critic with Static Spectral Risk Measures for Online and Offline Reinforcement Learning](https://mlsanity.com/paper/2507.03900/) (2025)
- [Risk-Averse MDPs under Reward Ambiguity](https://mlsanity.com/paper/2301.01045/) (2023)
- [Risk-Sensitive Q-Learning in Continuous Time with Application to Dynamic Portfolio Selection](https://mlsanity.com/paper/2512.02386/) (2025)

## Abstract (arXiv)

We study the minimization of a spectral risk measure of the total discounted cost generated by a Markov Decision Process (MDP) over a finite or infinite planning horizon. The MDP is assumed to have Borel state and action spaces and the cost function may be unbounded above. The optimization problem is split into two minimization problems using an infimum representation for spectral risk measures. We show that the inner minimization problem can be solved as an ordinary MDP on an extended state space and give sufficient conditions under which an optimal policy exists. Regarding the infinite dimensional outer minimization problem, we prove the existence of a solution and derive an algorithm for its numerical approximation. Our results include the findings in B"auerle and Ott (2011) in the special case that the risk measure is Expected Shortfall. As an application, we present a dynamic extension of the classical static optimal reinsurance problem, where an insurance company minimizes its cost of capital.