# Risk Aversion in Finite Markov Decision Processes Using Total Cost Criteria and Average Value at Risk

arXiv: https://arxiv.org/abs/1602.05130 · HTML: https://mlsanity.com/paper/1602.05130/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2016.

## Built on by (most cited first, 9 of 9)

- [Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments](https://mlsanity.com/paper/2504.19139/) (2025)
- [Risk-Averse Stochastic Shortest Path Planning](https://mlsanity.com/paper/2103.14727/) (2021)
- [Risk-aware Stochastic Shortest Path](https://mlsanity.com/paper/2203.01640/) (2022)
- [Risk-averse Total-reward MDPs with ERM and EVaR](https://mlsanity.com/paper/2408.17286/) (2024)
- [Risk-Averse Decision Making Under Uncertainty](https://mlsanity.com/paper/2109.04082/) (2021)
- [Risk-Averse MDPs under Reward Ambiguity](https://mlsanity.com/paper/2301.01045/) (2023)
- [Forward-PECVaR Algorithm: Exact Evaluation for CVaR SSPs](https://mlsanity.com/paper/2303.00672/) (2023)
- [R2L: Reliable Reinforcement Learning: Guaranteed Return & Reliable Policies in Reinforcement Learning](https://mlsanity.com/paper/2510.18074/) (2025)
- [Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modeling](https://mlsanity.com/paper/2610.14913/) (2026)

## Abstract (arXiv)

In this paper we present an algorithm to compute risk averse policies in Markov Decision Processes (MDP) when the total cost criterion is used together with the average value at risk (AVaR) metric. Risk averse policies are needed when large deviations from the expected behavior may have detrimental effects, and conventional MDP algorithms usually ignore this aspect. We provide conditions for the structure of the underlying MDP ensuring that approximations for the exact problem can be derived and solved efficiently. Our findings are novel inasmuch as average value at risk has not previously been considered in association with the total cost criterion. Our method is demonstrated in a rapid deployment scenario, whereby a robot is tasked with the objective of reaching a target location within a temporal deadline where increased speed is associated with increased probability of failure. We demonstrate that the proposed algorithm not only produces a risk averse policy reducing the probability of exceeding the expected temporal deadline, but also provides the statistical distribution of costs, thus offering a valuable analysis tool.