mlsanityData to 14 Oct 2026

Deep Exploration via Bootstrapped DQN

arXiv 1602.04621 ↗First posted 15 Feb 2016cs.LG cs.AI cs.SYcited by 527

Not on the map. Why?

Cited by 130 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computationally and statistically efficient manner through use of randomized value functions. Unlike dithering strategies such as epsilon-greedy exploration, bootstrapped DQN carries out temporally-extended (or deep) exploration; this can lead to exponentially faster learning. We demonstrate these benefits in complex stochastic MDPs and in the large-scale Arcade Learning Environment. Bootstrapped DQN substantially improves learning times and performance across most Atari games.

Key facts. Cited by 527 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 4; 2020: 4; 2021: 107; 2022: 79; 2023: 90; 2024: 73; 2025: 96; 2026: 73.

527 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2023Mastering Diverse Domains through World Modelscited by 1,253
  4. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  5. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  6. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 284
  7. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 214
  8. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  9. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  10. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  11. 2018Self-Imitation Learningcited by 138
  12. 2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learningcited by 126
38 more
  1. 2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
  2. 2021Epistemic Neural Networkscited by 120
  3. 2022On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networkscited by 120
  4. 2024To Believe or Not to Believe Your LLMcited by 120
  5. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 114
  6. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  7. 2021DEUP: Direct Epistemic Uncertainty Predictioncited by 104
  8. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 97
  9. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  10. 2021Reward is enough for convex MDPscited by 84
  11. 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothingcited by 84
  12. 2025Outcome-based Exploration for LLM Reasoningcited by 83
  13. 2024Simplifying Deep Temporal Difference Learningcited by 78
  14. 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 76
  15. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  16. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 65
  17. 2023Generative Flow Networks as Entropy-Regularized RLcited by 64
  18. 2022Reward Uncertainty for Exploration in Preference-based Reinforcement Learningcited by 61
  19. 2021When and How Mixup Improves Calibrationcited by 60
  20. 2021Reinforcement Learning, Bit by Bitcited by 59
  21. 2024Mitigating LLM Hallucinations via Conformal Abstentioncited by 59
  22. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  23. 2021On The Effect of Auxiliary Tasks on Representation Dynamicscited by 57
  24. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  25. 2023Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimizationcited by 56
  26. 2021Tactical Optimism and Pessimism for Deep Reinforcement Learningcited by 55
  27. 2024Feedback Efficient Online Fine-Tuning of Diffusion Modelscited by 53
  28. 2023Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Methodcited by 51
  29. 2021Randomized Exploration for Reinforcement Learning with General Value Function Approximationcited by 49
  30. 2021Quantifying Uncertainty in Deep Spatiotemporal Forecastingcited by 48
  31. 2024Efficient Exploration for LLMscited by 45
  32. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learningcited by 44
  33. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  34. 2022A Review of Uncertainty for Deep Reinforcement Learningcited by 40
  35. 2022Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priorscited by 39
  36. 2021Learning more skills through optimistic explorationcited by 38
  37. 2019An Optimistic Perspective on Offline Reinforcement Learningcited by 37
  38. 2022Faithful Heteroscedastic Regression with Neural Networkscited by 37

50 most cited of 527 All 527 (JSON)