# Risk-Averse Decision Making Under Uncertainty

arXiv: https://arxiv.org/abs/2109.04082 · HTML: https://mlsanity.com/paper/2109.04082/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 67 references that are in the map.
- First posted on arXiv 9 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach](https://mlsanity.com/paper/1506.02188/) (2015)
- [Algorithms for CVaR Optimization in MDPs](https://mlsanity.com/paper/1406.3339/) (2014)
- [Policy Gradient for Coherent Risk Measures](https://mlsanity.com/paper/1502.03919/) (2015)
- [How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics](https://mlsanity.com/paper/1710.11040/) (2017)
- [Disciplined Convex-Concave Programming](https://mlsanity.com/paper/1604.02639/) (2016)
- [STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation](https://mlsanity.com/paper/2103.02828/) (2021)
- [Policy Gradients for CVaR-Constrained MDPs](https://mlsanity.com/paper/1405.2690/) (2014)
- [Constrained Risk-Averse Markov Decision Processes](https://mlsanity.com/paper/2012.02423/) (2020)
- [Risk-sensitive safety analysis using Conditional Value-at-Risk](https://mlsanity.com/paper/2101.12086/) (2021)
- [Risk-Averse Planning Under Uncertainty](https://mlsanity.com/paper/1909.12499/) (2019)
- [Stochastic Finite State Control of POMDPs with LTL Specifications](https://mlsanity.com/paper/2001.07679/) (2020)
- [Stochastic Approximation for Risk-aware Markov Decision Processes](https://mlsanity.com/paper/1805.04238/) (2018)
- [Risk Aversion in Finite Markov Decision Processes Using Total Cost Criteria and Average Value at Risk](https://mlsanity.com/paper/1602.05130/) (2016)
- [Risk-Averse Stochastic Shortest Path Planning](https://mlsanity.com/paper/2103.14727/) (2021)
- [Risk-Sensitive Motion Planning using Entropic Value-at-Risk](https://mlsanity.com/paper/2011.11211/) (2020)
- [A Framework for Time-Consistent, Risk-Sensitive Model Predictive Control: Theory and Algorithms](https://mlsanity.com/paper/1703.01029/) (2017)
- [Control Theory Meets POMDPs: A Hybrid Systems Approach](https://mlsanity.com/paper/1905.08095/) (2019)
- [Risk-averse model predictive control](https://mlsanity.com/paper/1704.00342/) (2017)
- [Portfolio Optimization with Entropic Value-at-Risk](https://mlsanity.com/paper/1708.05713/) (2017)
- [Time-Optimal Navigation in Uncertain Environments with High-Level Specifications](https://mlsanity.com/paper/2103.01476/) (2021)

## Built on by (most cited first, 2 of 2)

- [RASR: Risk-Averse Soft-Robust MDPs with EVaR and Entropic Risk](https://mlsanity.com/paper/2209.04067/) (2022)
- [Optimal Transport-Guided Safety in Temporal Difference Reinforcement Learning](https://mlsanity.com/paper/2502.16328/) (2025)

## Abstract (arXiv)

A large class of decision making under uncertainty problems can be described via Markov decision processes (MDPs) or partially observable MDPs (POMDPs), with application to artificial intelligence and operations research, among others. Traditionally, policy synthesis techniques are proposed such that a total expected cost or reward is minimized or maximized. However, optimality in the total expected cost sense is only reasonable if system behavior in the large number of runs is of interest, which has limited the use of such policies in practical mission-critical scenarios, wherein large deviations from the expected behavior may lead to mission failure. In this paper, we consider the problem of designing policies for MDPs and POMDPs with objectives and constraints in terms of dynamic coherent risk measures, which we refer to as the constrained risk-averse problem. For MDPs, we reformulate the problem into a infsup problem via the Lagrangian framework and propose an optimization-based method to synthesize Markovian policies. For MDPs, we demonstrate that the formulated optimization problems are in the form of difference convex programs (DCPs) and can be solved by the disciplined convex-concave programming (DCCP) framework. We show that these results generalize linear programs for constrained MDPs with total discounted expected costs and constraints. For POMDPs, we show that, if the coherent risk measures can be defined as a Markov risk transition mapping, an infinite-dimensional optimization can be used to design Markovian belief-based policies. For stochastic finite-state controllers (FSCs), we show that the latter optimization simplifies to a (finite-dimensional) DCP and can be solved by the DCCP framework. We incorporate these DCPs in a policy iteration algorithm to design risk-averse FSCs for POMDPs.