mlsanityData to 7 Oct 2026

No Regrets for Learning the Prior in Bandits

arXiv 2107.06196 ↗First posted 13 Jul 2021cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

We propose tt AdaTS, a Thompson sampling algorithm that adapts sequentially to bandit tasks that it interacts with. The key idea in tt AdaTS is to adapt to an unknown task prior distribution by maintaining a distribution over its parameters. When solving a bandit task, that uncertainty is marginalized out and properly accounted for. tt AdaTS is a fully-Bayesian algorithm that can be implemented efficiently in several classes of bandit problems. We derive upper bounds on its Bayes regret that quantify the loss due to not knowing the task prior, and show that it is small. Our theory is supported by experiments, where tt AdaTS outperforms prior algorithms and works well even in challenging real-world problems.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 22 of its 48 references that are in the map. First posted on arXiv 13 Jul 2021.

Builds on

22 of 48 references.

References · 22

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  3. 2017A Tutorial on Thompson Samplingcited by 420
  4. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  5. 2013Learning to Optimize Via Posterior Samplingcited by 253
  6. 2014Online Clustering of Banditscited by 99
  7. 2014Combinatorial Multi-Armed Bandit and Its Extension to Probabilistically Triggered Armscited by 78
  8. 2014Algorithms for multi-armed bandit problemscited by 63
  9. 2021Meta-Thompson Samplingcited by 62
  10. 2019Information-Theoretic Confidence Bounds for Reinforcement Learningcited by 51
  11. 2017Multi-Task Learning for Contextual Banditscited by 47
  12. 2020Meta-learning with Stochastic Linear Banditscited by 45
10 more
  1. 2020Latent Bandits Revisitedcited by 43
  2. 2019Meta Dynamic Pricing: Transfer Learning Across Experimentscited by 41
  3. 2015Combinatorial Cascading Banditscited by 40
  4. 2013Sequential Transfer in Multi-armed Bandit with Finite Set of Modelscited by 38
  5. 2014Efficient Learning in Large-Scale Combinatorial Semi-Banditscited by 36
  6. 2014Matroid Bandits: Fast Combinatorial Optimization with Learningcited by 29
  7. 2020Policy Gradient Optimization of Thompson Sampling Policiescited by 8
  8. 2019Empirical Bayes Regret Minimizationcited by 6
  9. 2020Differentiable Linear Bandit Algorithmcited by 5
  10. 2020Meta-Learning Bandit Policies by Gradient Ascentcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 12; 2023: 5; 2024: 5; 2025: 5; 2026: 1.

29 papers.

Sort:

By area:

  1. 2021Hierarchical Bayesian Banditscited by 33
  2. 2022Multi-task Representation Learning with Stochastic Linear Banditscited by 20
  3. 2022Deep Hierarchy in Banditscited by 18
  4. 2023Impatient Bandits: Optimizing Recommendations for the Long-Term Without Delaycited by 17
  5. 2022Mixed-Effect Thompson Samplingcited by 14
  6. 2022Meta-Learning for Simple Regret Minimizationcited by 10
  7. 2022Towards Scalable and Robust Structured Bandits: A Meta-Learning Frameworkcited by 10
  8. 2022Hierarchical Conversational Preference Elicitation with Bandit Feedbackcited by 10
  9. 2022Multi-Task Off-Policy Learning from Bandit Feedbackcited by 9
  10. 2023Meta-Learning Adversarial Bandit Algorithmscited by 8
  11. 2023Thompson Sampling with Diffusion Generative Priorcited by 7
  12. 2024reBandit: Random Effects based Online RL algorithm for Reducing Cannabis Usecited by 6
17 more
  1. 2022Meta-Learning Hypothesis Spaces for Sequential Decision-makingcited by 5
  2. 2022Meta-Learning Adversarial Banditscited by 5
  3. 2024Diffusion Models Meet Contextual Banditscited by 5
  4. 2025Contextual Thompson Sampling via Generation of Missing Datacited by 5
  5. 2022Robust Contextual Linear Banditscited by 4
  6. 2024Prior-Dependent Allocations for Bayesian Fixed-Budget Best-Arm Identification in Structured Banditscited by 4
  7. 2024Meta Learning in Bandits within Shared Affine Subspacescited by 4
  8. 2022Thompson Sampling for Robust Transfer in Multi-Task Banditscited by 3
  9. 2022Lifelong Bandit Optimization: No Prior and No Regretcited by 3
  10. 2024Online Posterior Sampling with a Diffusion Priorcited by 2
  11. 2025Adaptive Prior Selection in Gaussian Process Bandits with Thompson Samplingcited by 1
  12. 2025A Classification View on Meta Learning Banditscited by 1
  13. 2023Finite-Time Logarithmic Bayes Regret Upper Boundscited by 0
  14. 2023Interactive Graph Convolutional Filteringcited by 0
  15. 2025Representative Action Selection for Large Action Space Bandit Familiescited by 0
  16. 2025Co-Exploration and Co-Exploitation via Shared Structure in Multi-Task Banditscited by 0
  17. 2026Meta-Learning for Repeated Bayesian Persuasioncited by 0