mlsanityData to 7 Oct 2026

Implicit Quantile Networks for Distributional Reinforcement Learning

arXiv 1806.06923 ↗First posted 14 Jun 2018cs.LG cs.AI stat.MLcited by 362

Not on the map. Why?

Cited by 36 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this work, we build on recent advances in distributional reinforcement learning to give a generally applicable, flexible, and state-of-the-art distributional variant of DQN. We achieve this by using quantile regression to approximate the full quantile function for the state-action return distribution. By reparameterizing a distribution over the sample space, this yields an implicitly defined return distribution and gives rise to a large class of risk-sensitive policies. We demonstrate improved performance on the 57 Atari 2600 games in the ALE, and use our algorithm's implicitly defined distributions to study the effects of risk-sensitive policies in Atari games.

Key facts. Cited by 362 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 4; 2021: 59; 2022: 60; 2023: 64; 2024: 52; 2025: 56; 2026: 66.

362 papers.

Sort:

By area:

  1. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  2. 2023Mastering Diverse Domains through World Modelscited by 1,193
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  4. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  5. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  6. 2022On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networkscited by 119
  7. 2021Learning to drive from a world on railscited by 114
  8. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 113
  9. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 104
  10. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  11. 2021Conservative Offline Distributional Reinforcement Learningcited by 78
  12. 2021Risk-Averse Offline Reinforcement Learningcited by 65
38 more
  1. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 63
  2. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 59
  3. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 59
  4. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  5. 2021Tactical Optimism and Pessimism for Deep Reinforcement Learningcited by 55
  6. 2021Muesli: Combining Improvements in Policy Optimizationcited by 53
  7. 2022Atari-5: Distilling the Arcade Learning Environment down to Five Gamescited by 44
  8. 2021Universal Off-Policy Evaluationcited by 43
  9. 2021MICo: Improved representations via sampling-based state similarity for Markov decision processescited by 43
  10. 2020Overcoming Model Bias for Robust Offline Deep Reinforcement Learningcited by 42
  11. 2021Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 42
  12. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  13. 2022Efficient Risk-Averse Reinforcement Learningcited by 41
  14. 2022Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Riskcited by 40
  15. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 38
  16. 2019An Optimistic Perspective on Offline Reinforcement Learningcited by 37
  17. 2023Small batch deep reinforcement learningcited by 35
  18. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  19. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 30
  20. 2023An Analysis of Quantile Temporal-Difference Learningcited by 30
  21. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  22. 2021Off-Policy Risk Assessment in Contextual Banditscited by 29
  23. 2023Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlocited by 29
  24. 2022On the Generalization of Representations in Reinforcement Learningcited by 28
  25. 2023Distributional GFlowNets with Quantile Flowscited by 28
  26. 2022MuZero with Self-competition for Rate Control in VP9 Video Compressioncited by 27
  27. 2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
  28. 2023The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement Learningcited by 27
  29. 2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
  30. 2021Active Offline Policy Selectioncited by 26
  31. 2022Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 26
  32. 2021Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmarkcited by 23
  33. 2024More Benefits of Being Distributional: Second-Order Bounds for Reinforcement Learningcited by 23
  34. 2022Addressing Optimism Bias in Sequence Modeling for Reinforcement Learningcited by 22
  35. 2023Distributional Soft Actor-Critic with Three Refinementscited by 22
  36. 2021Distributional Reinforcement Learning for Multi-Dimensional Reward Functionscited by 21
  37. 2022Solving Continuous Control via Q-learningcited by 21
  38. 2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 21

50 most cited of 362 All 362 (JSON)