mlsanityData to 14 Oct 2026

Value of Information-Enhanced Exploration in Bootstrapped DQN

arXiv 2511.02969 ↗First posted 4 Nov 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Efficient exploration in deep reinforcement learning remains a fundamental challenge, especially in environments characterized by high-dimensional states and sparse rewards. Traditional exploration strategies that rely on random local policy noise, such as ε-greedy and Boltzmann exploration methods, often struggle to efficiently balance exploration and exploitation. In this paper, we integrate the notion of (expected) value of information (EVOI) within the well-known Bootstrapped DQN algorithmic framework, to enhance the algorithm's deep exploration ability. Specifically, we develop two novel algorithms that incorporate the expected gain from learning the value of information into Bootstrapped DQN. Our methods use value of information estimates to measure the discrepancies of opinions among distinct network heads, and drive exploration towards areas with the most potential. We evaluate our algorithms with respect to performance and their ability to exploit inherent uncertainty arising from random network initialization. Our experiments in complex, sparse-reward Atari games demonstrate increased performance, all the while making better use of uncertainty, and, importantly, without introducing extra hyperparameters.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 34 references that are in the map. First posted on arXiv 4 Nov 2025.

Builds on

16 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024The Curse of Diversity in Ensemble-Based Exploration7 May

Other references · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2018Exploration by Random Network Distillationcited by 993
  4. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  5. 2016Deep Exploration via Bootstrapped DQNcited by 527
  6. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  7. 2019Self-Supervised Exploration via Disagreementcited by 265
  8. 2018An Introduction to Deep Reinforcement Learningcited by 206
  9. 2014Learning to Optimize via Information-Directed Samplingcited by 168
  10. 2017Deep Exploration via Randomized Value Functionscited by 156
  11. 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 152
  12. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
3 more
  1. 2018Information-Directed Exploration for Deep Reinforcement Learningcited by 52
  2. 2021Diversity and Generalization in Neural Network Ensemblescited by 32
  3. 2013Model-Based Bayesian Explorationcited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Information-Based Exploration via Random Features for Reinforcement Learning
  2. 2025β-DQN: Improving Deep Q-Learning By Evolving the Behavior
  3. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization
  4. 2024Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search
  5. 2025DQN Performance with Epsilon Greedy Policies and Prioritized Experience Replay
  6. 2025On Efficient Bayesian Exploration in Model-Based Reinforcement Learning
  7. 2026Value Bonuses using Ensemble Errors for Exploration in Reinforcement Learning
  8. 2024VDSC: Enhancing Exploration Timing with Value Discrepancy and State Counts