mlsanityData to 7 Oct 2026

Bootstrapped Thompson Sampling and Deep Exploration

arXiv 1507.00300 ↗First posted 1 Jul 2015stat.ML cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

This technical note presents a new approach to carrying out the kind of exploration achieved by Thompson sampling, but without explicitly maintaining or sampling from posterior distributions. The approach is based on a bootstrap technique that uses a combination of observed and artificially generated data. The latter serves to induce a prior distribution which, as we will demonstrate, is critical to effective exploration. We explain how the approach can be applied to multi-armed bandit and reinforcement learning problems and how it relates to Thompson sampling. The approach is particularly well-suited for contexts in which exploration is coupled with deep learning, since in these settings, maintaining or generating samples from a posterior distribution becomes computationally infeasible.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jul 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 9; 2023: 5; 2024: 6; 2025: 7; 2026: 1.

40 papers.

Sort:

By area:

  1. 2021Epistemic Neural Networkscited by 120
  2. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  3. 2024Distinguishing the Knowable from the Unknowable with Language Modelscited by 45
  4. 2023Approximate Thompson Sampling via Epistemic Neural Networkscited by 31
  5. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  6. 2021From Predictions to Decisions: The Importance of Joint Predictive Distributionscited by 28
  7. 2021The Neural Testbed: Evaluating Joint Predictionscited by 22
  8. 2022From Dirichlet to Rubin: Optimistic Exploration in RL without Bonusescited by 22
  9. 2022Fine-Tuning Language Models via Epistemic Neural Networkscited by 22
  10. 2021Diffusion Approximations for Thompson Sampling in the Small Gap Regimecited by 12
  11. 2022Evaluating High-Order Predictive Distributions in Deep Learningcited by 10
  12. 2023Differential Good Arm Identificationcited by 8
28 more
  1. 2025Exploration-Driven Generative Interactive Environmentscited by 8
  2. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8
  3. 2022Residual Bootstrap Exploration for Stochastic Linear Banditcited by 7
  4. 2024Q-Star Meets Scalable Posterior Sampling: Bridging Theory and Practice via HyperAgentcited by 6
  5. 2021Risk Aware and Multi-Objective Decision Making with Distributional Monte Carlo Tree Searchcited by 5
  6. 2021State-Aware Variational Thompson Sampling for Deep Q-Networkscited by 5
  7. 2022SOTIF Entropy: Online SOTIF Risk Quantification and Mitigation for Autonomous Drivingcited by 5
  8. 2024Model-Free Active Exploration in Reinforcement Learningcited by 5
  9. 2025Contextual Thompson Sampling via Generation of Missing Datacited by 5
  10. 2026Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directionscited by 5
  11. 2021From Optimality to Robustness: Dirichlet Sampling Strategies in Stochastic Banditscited by 4
  12. 2025Addressing Missing Data Issue for Diffusion-based Recommendationcited by 4
  13. 2021Debiasing Samples from Online Learning Using Bootstrapcited by 3
  14. 2022Improving the Diversity of Bootstrapped DQN by Replacing Priors With Noisecited by 3
  15. 2023Multiplier Bootstrap-based Explorationcited by 3
  16. 2023State Representation Learning Using an Unbalanced Atlascited by 3
  17. 2021CORe: Capitalizing On Rewards in Bandit Explorationcited by 1
  18. 2022The Implicit Delta Methodcited by 1
  19. 2024Epinet for Content Cold Startcited by 1
  20. 2021GuideBoot: Guided Bootstrap for Deep Contextual Banditscited by 0
  21. 2021Knowledge is reward: Learning optimal exploration by predictive reward cashingcited by 0
  22. 2022A Deep Bayesian Bandits Approach for Anticancer Therapy: Exploration via Functional Priorcited by 0
  23. 2022A Nonparametric Contextual Bandit with Arm-level Eligibility Control for Customer Service Routingcited by 0
  24. 2024Variation Due to Regularization Tractably Recovers Bayesian Deep Learningcited by 0
  25. 2024Bayesian Bandit Algorithms with Approximate Inference in Stochastic Linear Banditscited by 0
  26. 2025EVaDE : Event-Based Variational Thompson Sampling for Model-Based Reinforcement Learningcited by 0
  27. 2025Leveraging priors on distribution functions for multi-arm banditscited by 0
  28. 2025Enhancing Q-Value Updates in Deep Q-Learning via Successor-State Predictioncited by 0