# Prospect-Theory Behavior from Bellman Optimality in MDPs with Catastrophic States

arXiv: https://arxiv.org/abs/2606.00970 · HTML: https://mlsanity.com/paper/2606.00970/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 20 references that are in the map.
- First posted on arXiv 31 May 2026.

## Builds on (most cited first, 1 of 1)

- [Risk aversion as an evolutionary adaptation](https://mlsanity.com/paper/1310.6338/) (2013)

## Abstract (arXiv)

We study risk-neutral control in Markov decision processes with an absorbing catastrophic state. Even though rewards are linear and the agent has no utility curvature, probability weighting, or framing dependence, standard Bellman optimality produces three prospect-theory-like signatures: an S-shaped value-function profile (convex near catastrophe, concave in the far field), an endogenous loss-sensitivity coefficient λ^*(S) > 1, and a reflection-effect policy reversal. Across 495 configurations, the optimal policy plays safe near catastrophe in positive-drift (growth) regimes despite the risky action's higher immediate expected value, and plays risky near catastrophe in negative-drift (decline) regimes despite the safe action's lower immediate expected loss. We derive a closed-form expression for the asymptotic loss-aversion plateau barλ that depends only on win probability p, payoff asymmetry r = |Deltaℓ/Deltaw|, and discount factor β, and matches numerical solutions to R2 = 0.999. The mechanism does not require asymmetric payoffs. Across a sweep of (p,β) at three asymmetry levels, the asymmetry share of barλ above unity has median 4.6% at r = 1.25 and rises to 13.9% at r = 2, with the boundary contribution exceeding the asymmetry contribution in every cell tested. The phenomena persist under tabular Q-learning (a model-free agent reproduces V^* at correlation 0.98 in growth and 1.00 in decline) and under stochastic transitions with Gaussian, heavy-tailed Student-t3, and asymmetric skew-normal noise up to 50% of the step size, where the asymptotic plateau tracks the closed-form prediction within 0.41% for safe-channel noise and within 9.6% for risky-channel or both-channel noise. These results identify absorbing failure states as a sufficient structural mechanism for prospect-theory-like behavior under optimal control.