mlsanityData to 14 Oct 2026

Hierarchical Bayesian Bandits

arXiv 2111.06929 ↗First posted 12 Nov 2021cs.LG cs.AIcited by 35

Not on the map. Why?

Abstract (arXiv)

Meta-, multi-task, and federated learning can be all viewed as solving similar tasks, drawn from a distribution that reflects task similarities. We provide a unified view of all these problems, as learning to act in a hierarchical Bayesian bandit. We propose and analyze a natural hierarchical Thompson sampling algorithm (HierTS) for this class of problems. Our regret bounds hold for many variants of the problems, including when the tasks are solved sequentially or in parallel; and show that the regret decreases with a more informative prior. Our proofs rely on a novel total variance decomposition that can be applied beyond our models. Our theory is complemented by experiments, which show that the hierarchy helps with knowledge sharing among the tasks. This confirms that hierarchical Bayesian bandits are a universal and statistically-efficient tool for learning to act with similar bandit tasks.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 27 of its 51 references that are in the map. First posted on arXiv 12 Nov 2021.

Builds on

27 of 51 references.

References · 27

  1. 2017A Survey on Multi-Task Learningcited by 840
  2. 2017A Tutorial on Thompson Samplingcited by 433
  3. 2013Learning to Optimize Via Posterior Samplingcited by 256
  4. 2014An Information-Theoretic Analysis of Thompson Samplingcited by 178
  5. 2014Online Clustering of Banditscited by 102
  6. 2014Combinatorial Multi-Armed Bandit and Its Extension to Probabilistically Triggered Armscited by 79
  7. 2015Cascading Bandits: Learning to Rank in the Cascade Modelcited by 77
  8. 2019Meta-learning of Sequential Strategiescited by 70
  9. 2021Meta-Thompson Samplingcited by 63
  10. 2021Federated Multi-Armed Banditscited by 59
  11. 2019Information-Theoretic Confidence Bounds for Reinforcement Learningcited by 51
  12. 2017Multi-Task Learning for Contextual Banditscited by 47
15 more
  1. 2020Meta-learning with Stochastic Linear Banditscited by 46
  2. 2016On Context-Dependent Clustering of Banditscited by 44
  3. 2020Latent Bandits Revisitedcited by 44
  4. 2021Bayesian decision-making under misspecified priors with applications to meta-learningcited by 44
  5. 2019Meta Dynamic Pricing: Transfer Learning Across Experimentscited by 42
  6. 2013Sequential Transfer in Multi-armed Bandit with Finite Set of Modelscited by 38
  7. 2021No Regrets for Learning the Prior in Banditscited by 30
  8. 2016Contextual Bandits with Latent Confounders: An NMF Approachcited by 26
  9. 2021Metadata-based Multi-Task Bandits with Bayesian Hierarchical Modelscited by 24
  10. 2016Stochastic Rank-1 Banditscited by 23
  11. 2020Kernel Methods for Cooperative Multi-Agent Contextual Banditscited by 21
  12. 2016DCM Bandits: Learning to Rank with Multiple Clickscited by 17
  13. 2021Thompson Sampling with a Mixture Priorcited by 16
  14. 2021Metalearning Linear Bandits by Prior Updatecited by 16
  15. 2020Multitask Bandit Learning Through Heterogeneous Feedback Aggregationcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 13; 2023: 4; 2024: 8; 2025: 5; 2026: 5.

35 papers.

Sort:

By area:

  1. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  2. 2022Deep Hierarchy in Banditscited by 19
  3. 2022Mixed-Effect Thompson Samplingcited by 14
  4. 2022Bayesian Fixed-Budget Best-Arm Identificationcited by 13
  5. 2023Online Clustering of Bandits with Misspecified User Modelscited by 12
  6. 2022Towards Scalable and Robust Structured Bandits: A Meta-Learning Frameworkcited by 11
  7. 2022Meta-Learning for Simple Regret Minimizationcited by 10
  8. 2022Hierarchical Conversational Preference Elicitation with Bandit Feedbackcited by 10
  9. 2024Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learningcited by 10
  10. 2022Multi-Task Off-Policy Learning from Bandit Feedbackcited by 9
  11. 2022Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variablescited by 7
  12. 2024Meta Clustering of Neural Banditscited by 7
23 more
  1. 2025Survey: Multi-Armed Bandits Meet Large Language Modelscited by 7
  2. 2024reBandit: Random Effects based Online RL algorithm for Reducing Cannabis Usecited by 6
  3. 2023Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learningcited by 5
  4. 2024Diffusion Models Meet Contextual Banditscited by 5
  5. 2022Multi-Environment Meta-Learning in Stochastic Linear Banditscited by 4
  6. 2022Robust Contextual Linear Banditscited by 4
  7. 2022Lifelong Bandit Optimization: No Prior and No Regretcited by 4
  8. 2024Prior-Dependent Allocations for Bayesian Fixed-Budget Best-Arm Identification in Structured Banditscited by 4
  9. 2024Federated Linear Contextual Bandits with Heterogeneous Clientscited by 4
  10. 2022Thompson Sampling for Robust Transfer in Multi-Task Banditscited by 3
  11. 2023Only Pay for What Is Uncertain: Variance-Adaptive Thompson Samplingcited by 3
  12. 2024Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysiscited by 2
  13. 2024Online Posterior Sampling with a Diffusion Priorcited by 2
  14. 2025Adaptive Prior Selection in Gaussian Process Bandits with Thompson Samplingcited by 2
  15. 2025A Classification View on Meta Learning Banditscited by 1
  16. 2025Empirical Bayesian Multi-Bandit Learningcited by 1
  17. 2023Finite-Time Logarithmic Bayes Regret Upper Boundscited by 0
  18. 2025On the optimal regret of collaborative personalized linear banditscited by 0
  19. 2026Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Riskcited by 0
  20. 2026Manifold Bandits: Bayesian Curriculum Learning over the Latent Geometry of Large Language Modelscited by 0
  21. 2026Leveraging Similarities in Multi-Armed Banditscited by 0
  22. 2026Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendationcited by 0
  23. 2026Online LLM Personalization from Pairwise Preferences Across Userscited by 0