mlsanityData to 14 Oct 2026

Quantile Geometry Regularization for Distributional Reinforcement Learning

arXiv 2605.08182 ↗First posted 5 May 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Quantile-based distributional reinforcement learning methods learn return distributions through sampled quantile regression, but their bootstrapped target quantiles may induce distorted or degenerate distribution estimates. We propose Robust Quantile-based Implicit Quantile Networks (RQIQN), a lightweight Wasserstein distributionally robust enhancement boosted from a quantile estimation perspective. We first reinterpret a snapshot of IQN loss as a collection of local empirical quantile estimation problems over sampled current fractions. We then robustify each local slot with a Wasserstein distributionally robust quantile estimation formulation, yielding a closed-form, fraction-dependent correction to the Bellman target. This correction directly addresses distributional degeneration: its median antisymmetry preserves the risk-neutral quantile average, while its monotonicity enlarges upper-lower quantile gaps and counteracts collapsed distributional spread. RQIQN thus regularizes quantile geometry without changing the underlying value objective or requiring additional sample set reconstruction. Finally, we empirically show that the proposed RQIQN outperforms other existing quantile-based distributional reinforcement learning algorithms in risk-sensitive navigation and Atari games.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 25 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

15 of 25 references.

References · 15

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  3. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  4. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  5. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
  6. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  7. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
  8. 2019Fully Parameterized Quantile Function for Distributional Reinforcement Learningcited by 88
  9. 2019Statistics and Samples in Distributional Reinforcement Learningcited by 63
  10. 2022Atari-5: Distilling the Arcade Learning Environment down to Five Gamescited by 44
  11. 2023Robust Quadrupedal Locomotion via Risk-Averse Policy Learningcited by 19
  12. 2020Distributional Reinforcement Learning via Moment Matchingcited by 10
3 more
  1. 2023Robust Unmanned Surface Vehicle Navigation with Distributional Reinforcement Learningcited by 8
  2. 2025Perturbation-mitigated USV Navigation with Distributionally Robust Reinforcement Learningcited by 1
  3. 2026Wasserstein Distributionally Robust Quantile Regressioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning
  2. 2026Statistical Efficiency and Inference of Quantile Distributional Reinforcement Learning
  3. 2026Deep Reinforcement Learning with Buffered Quantile Objectives
  4. 2024Tilted Quantile Gradient Updates for Quantile-Constrained Reinforcement Learning
  5. 2025Quantile Q-Learning: Revisiting Offline Extreme Q-Learning with Quantile Regression
  6. 2026Distorted Distributional Policy Evaluation for Offline Reinforcement Learning
  7. 2025Deep Distributional Learning with Non-crossing Quantile Network
  8. 2024Echoes of Socratic Doubt: Embracing Uncertainty in Calibrated Evidential Reinforcement Learning