mlsanityData to 7 Oct 2026

Fully Parameterized Quantile Function for Distributional Reinforcement Learning

arXiv 1911.02140 ↗First posted 5 Nov 2019cs.LG cs.AI stat.MLcited by 87

Not on the map. Why?

Abstract (arXiv)

Distributional Reinforcement Learning (RL) differs from traditional RL in that, rather than the expectation of total returns, it estimates distributions and has achieved state-of-the-art performance on Atari Games. The key challenge in practical distributional RL algorithms lies in how to parameterize estimated distributions so as to better approximate the true continuous distribution. Existing distributional RL algorithms parameterize either the probability side or the return value side of the distribution function, leaving the other side uniformly fixed as in C51, QR-DQN or randomly sampled as in IQN. In this paper, we propose fully parameterized quantile function that parameterizes both the quantile fraction axis (i.e., the x-axis) and the value axis (i.e., y-axis) for distributional RL. Our algorithm contains a fraction proposal network that generates a discrete set of quantile fractions and a quantile value network that gives corresponding quantile values. The two networks are jointly trained to find the best approximation of the true distribution. Experiments on 55 Atari Games show that our algorithm significantly outperforms existing distributional RL algorithms and creates a new record for the Atari Learning Environment for non-distributed agents.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 18; 2023: 21; 2024: 7; 2025: 10; 2026: 15.

87 papers.

Sort:

By area:

  1. 2021Conservative Offline Distributional Reinforcement Learningcited by 78
  2. 2021Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 42
  3. 2023An Analysis of Quantile Temporal-Difference Learningcited by 30
  4. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  5. 2023Distributional GFlowNets with Quantile Flowscited by 28
  6. 2023The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement Learningcited by 27
  7. 2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
  8. 2024More Benefits of Being Distributional: Second-Order Bounds for Reinforcement Learningcited by 23
  9. 2023Distributional Soft Actor-Critic with Three Refinementscited by 22
  10. 2021Distributional Reinforcement Learning for Multi-Dimensional Reward Functionscited by 21
  11. 2025AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisitioncited by 19
  12. 2021Probabilistic Time Series Forecasting with Implicit Quantile Networkscited by 17
38 more
  1. 2022Bridging Distributional and Risk-sensitive Reinforcement Learning with Provable Regret Boundscited by 16
  2. 2024Foundations of Multivariate Distributional Reinforcement Learningcited by 16
  3. 2021Non-decreasing Quantile Function Network with Efficient Exploration for Distributional Reinforcement Learningcited by 15
  4. 2021GMAC: A Distributional Perspective on Actor-Critic Frameworkcited by 15
  5. 2024A Distributional Analogue to the Successor Representationcited by 15
  6. 2023The Statistical Benefits of Quantile Temporal-Difference Learning for Value Estimationcited by 14
  7. 2021Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigationcited by 13
  8. 2022Deep Non-Crossing Quantiles through the Partial Derivativecited by 12
  9. 2023Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learningcited by 12
  10. 2026UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experiencecited by 11
  11. 2021Safe Distributional Reinforcement Learningcited by 10
  12. 2022Distributional Hamilton-Jacobi-Bellman Equations for Continuous-Time Reinforcement Learningcited by 9
  13. 2022The Nature of Temporal Difference Errors in Multi-step Distributional Reinforcement Learningcited by 9
  14. 2022Quantile Constrained Reinforcement Learning: A Reinforcement Learning Framework Constraining Outage Probabilitycited by 9
  15. 2021Measuring Progress in Deep Reinforcement Learning Sample Efficiencycited by 8
  16. 2022Distributional Reinforcement Learning with Regularized Wasserstein Losscited by 8
  17. 2023Diverse Projection Ensembles for Distributional Reinforcement Learningcited by 8
  18. 2021A Cram'er Distance perspective on Quantile Regression based Distributional Reinforcement Learningcited by 7
  19. 2022Reinforcement Learning with Heterogeneous Data: Estimation and Inferencecited by 7
  20. 2024Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Modelcited by 7
  21. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulatorcited by 7
  22. 2021MMD-MIX: Value Function Factorisation with Maximum Mean Discrepancy for Cooperative Multi-Agent Reinforcement Learningcited by 6
  23. 2021Enabling risk-aware Reinforcement Learning for medical interventions through uncertainty decompositioncited by 6
  24. 2022Autonomous Unmanned Aerial Vehicle Navigation using Reinforcement Learning: A Systematic Reviewcited by 6
  25. 2025Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learningcited by 6
  26. 2023Value-Distributional Model-Based Reinforcement Learningcited by 5
  27. 2025A Finite Sample Analysis of Distributional TD Learning with Linear Function Approximationcited by 5
  28. 2021Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learningcited by 4
  29. 2022How Does Return Distribution in Distributional Reinforcement Learning Help Optimization?cited by 4
  30. 2023Distributional Bellman Operators over Mean Embeddingscited by 4
  31. 2025Distributional Soft Actor-Critic with Diffusion Policycited by 4
  32. 2021Distributional Reinforcement Learning with Unconstrained Monotonic Neural Networkscited by 3
  33. 2022MCMARL: Parameterizing Value Function via Mixture of Categorical Distributions for Multi-Agent Reinforcement Learningcited by 3
  34. 2022A Simulation Environment and Reinforcement Learning Method for Waste Reductioncited by 3
  35. 2022Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Controlcited by 3
  36. 2022Sample-based Uncertainty Quantification with a Single Deterministic Neural Networkcited by 3
  37. 2023Multi-compartment Neuron and Population Encoding Powered Spiking Neural Network for Deep Distributional Reinforcement Learningcited by 3
  38. 2023Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterioncited by 3

50 most cited of 87 All 87 (JSON)