# Prospect-theoretic Q-learning

arXiv: https://arxiv.org/abs/2104.05311 · HTML: https://mlsanity.com/paper/2104.05311/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2021.

## Built on by (most cited first, 6 of 6)

- [Q-learning for Quantile MDPs: A Decomposition, Performance, and Convergence Analysis](https://mlsanity.com/paper/2410.24128/) (2024)
- [Privacy-Preserving Reinforcement Learning Beyond Expectation](https://mlsanity.com/paper/2203.10165/) (2022)
- [Beyond Rational Illusion: Behaviorally Realistic Strategic Classification](https://mlsanity.com/paper/2605.19674/) (2026)
- [Risk-Aware Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2304.02005/) (2023)
- [Teaching Precommitted Agents: Model-Free Policy Evaluation and Control in Quasi-Hyperbolic Discounted MDPs](https://mlsanity.com/paper/2509.06094/) (2025)
- [Choose Your Battles: Distributed Learning Over Multiple Tug of War Games](https://mlsanity.com/paper/2509.20147/) (2025)

## Abstract (arXiv)

We consider a prospect theoretic version of the classical Q-learning algorithm for discounted reward Markov decision processes, wherein the controller perceives a distorted and noisy future reward, modeled by a nonlinearity that accentuates gains and underrepresents losses relative to a reference point. We analyze the asymptotic behavior of the scheme by analyzing its limiting differential equation and using the theory of monotone dynamical systems to infer its asymptotic behavior. Specifically, we show convergence to equilibria, and establish some qualitative facts about the equilibria themselves.