Optimistic Reinforcement Learning with Quantile Objectives
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
Reinforcement Learning (RL) has achieved tremendous success in recent years. However, the classical foundations of RL do not account for the risk sensitivity of the objective function, which is critical in various fields, including healthcare and finance. A popular approach to incorporate risk sensitivity is to optimize a specific quantile of the cumulative reward distribution. In this paper, we develop UCB-QRL, an optimistic learning algorithm for the τ-quantile objective in finite-horizon Markov decision processes (MDPs). UCB-QRL is an iterative algorithm in which, at each iteration, we first estimate the underlying transition probability and then optimize the quantile value function over a confidence ball around this estimate. We show that UCB-QRL yields a high-probability regret bound mathcal Oleft((2/kappa)H+1H√SATHlog(2SATH/δ)right) in the episodic setting with S states, A actions, T episodes, and H horizons. Here, kappa>0 is a problem-dependent constant that captures the sensitivity of the underlying MDP's quantile value.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. First posted on arXiv 12 Nov 2025.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Risk-Sensitive Reinforcement Learning with Smoothed Quantile Objectives
- 2025Tail Distribution of Regret in Optimistic Reinforcement Learning
- 2024Optimistic Q-learning for average reward and episodic reinforcement learning
- 2024Tilted Quantile Gradient Updates for Quantile-Constrained Reinforcement Learning
- 2024Improved Regret Bound for Safe Reinforcement Learning via Tighter Cost Pessimism and Reward Optimism
- 2024Q-learning for Quantile MDPs: A Decomposition, Performance, and Convergence Analysis
- 2025Q-Learning with Shift-Aware Upper Confidence Bound in Non-Stationary Reinforcement Learning
- 2024A Reductions Approach to Risk-Sensitive Reinforcement Learning with Optimized Certainty Equivalents