# Long-Term Sequential Decision Making under Risk

arXiv: https://arxiv.org/abs/2607.19914 · HTML: https://mlsanity.com/paper/2607.19914/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Builds on (most cited first, 14 of 14)

- [Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach](https://mlsanity.com/paper/1506.02188/) (2015)
- [Reinforcement Learning with Dynamic Convex Risk Measures](https://mlsanity.com/paper/2112.13414/) (2021)
- [On Dynamic Programming Decompositions of Static Risk Measures in Markov Decision Processes](https://mlsanity.com/paper/2304.12477/) (2023)
- [Decision under Uncertainty](https://mlsanity.com/paper/1304.1527/) (2013)
- [Optimizing Return Distributions with Distributional Dynamic Programming](https://mlsanity.com/paper/2501.13028/) (2025)
- [Risk-Averse Stochastic Shortest Path Planning](https://mlsanity.com/paper/2103.14727/) (2021)
- [On the Global Convergence of Risk-Averse Natural Policy Gradient Methods with Expected Conditional Risk Measures](https://mlsanity.com/paper/2301.10932/) (2023)
- [Efficient Risk-sensitive Planning via Entropic Risk Measures](https://mlsanity.com/paper/2502.20423/) (2025)
- [Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning](https://mlsanity.com/paper/2407.07631/) (2024)
- [Planning for Risk-Aversion and Expected Value in MDPs](https://mlsanity.com/paper/2110.12746/) (2021)
- [Risk-Averse Reinforcement Learning via Dynamic Time-Consistent Risk Measures](https://mlsanity.com/paper/2301.05981/) (2023)
- [Recursive Entropic Risk Optimization in Discounted MDPs: Sample Complexity Bounds with a Generative Model](https://mlsanity.com/paper/2506.00286/) (2025)
- [Risk-averse Total-reward MDPs with ERM and EVaR](https://mlsanity.com/paper/2408.17286/) (2024)
- [Markov Decision Processes with Value-at-Risk Criterion](https://mlsanity.com/paper/2507.22355/) (2025)

## Abstract (arXiv)

We study finite-horizon MDP planning under root-based (resolute) risk objectives that apply a rank-dependent functional to the distribution of total returns. Such objectives are non-linear in the return distribution and generally break Bellman optimality, so direct optimization by scenario-tree enumeration is intractable. We propose ERQDP, an enumeration-free and sampling-free method that solves a rank--quantile surrogate via exact DP (Dynamic Programming), evaluates candidate policies exactly by DP over return Probability Mass Functions (PMFs) on a discretized return grid (with an explicit rounding bound), and refines the surrogate in an anytime loop that reports an explicit upper--lower gap (certificate) for the target objective up to discretization budgets. Across tested benchmarks, ERQDP returns certified solutions or explicit residual gaps, enables fast risk-parameter sweeps with substantial runtime gains, and supports both risk-averse and risk-seeking behaviors.