mlsanityData to 7 Oct 2026

Metadata-based Multi-Task Bandits with Bayesian Hierarchical Models

arXiv 2108.06422 ↗First posted 13 Aug 2021cs.LG cs.AIcited by 24

Not on the map. Why?

Abstract (arXiv)

How to explore efficiently is a central problem in multi-armed bandits. In this paper, we introduce the metadata-based multi-task bandit problem, where the agent needs to solve a large number of related multi-armed bandit tasks and can leverage some task-specific features (i.e., metadata) to share knowledge across tasks. As a general framework, we propose to capture task relations through the lens of Bayesian hierarchical models, upon which a Thompson sampling algorithm is designed to efficiently learn task relations, share information, and minimize the cumulative regrets. Two concrete examples for Gaussian bandits and Bernoulli bandits are carefully analyzed. The Bayes regret for Gaussian bandits clearly demonstrates the benefits of information sharing with our algorithm. The proposed method is further supported by extensive experiments.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 35 of its 70 references that are in the map. First posted on arXiv 13 Aug 2021.

Builds on

35 of 70 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016Variational Inference: A Review for Statisticianscited by 1,466
  3. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  4. 2017A Survey on Multi-Task Learningcited by 838
  5. 2017A Tutorial on Thompson Samplingcited by 420
  6. 2013Learning to Optimize Via Posterior Samplingcited by 253
  7. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  8. 2014Online Clustering of Banditscited by 99
  9. 2021Adapting to Misspecification in Contextual Banditscited by 81
  10. 2016Delay and Cooperation in Nonstochastic Banditscited by 71
  11. 2019Randomized Exploration in Generalized Linear Banditscited by 70
  12. 2021Meta-Thompson Samplingcited by 62
23 more
  1. 2017Markov Decision Processes with Continuous Side Informationcited by 54
  2. 2013A Gang of Banditscited by 49
  3. 2020Meta-learning with Stochastic Linear Banditscited by 45
  4. 2020Latent Bandits Revisitedcited by 43
  5. 2021Bayesian decision-making under misspecified priors with applications to meta-learningcited by 43
  6. 2019Meta Dynamic Pricing: Transfer Learning Across Experimentscited by 41
  7. 2013Sequential Transfer in Multi-armed Bandit with Finite Set of Modelscited by 38
  8. 2015A Survey on Contextual Multi-armed Banditscited by 34
  9. 2020Cooperative Multi-Agent Bandits with Heavy Tailscited by 31
  10. 2017Online Clustering of Contextual Cascading Banditscited by 24
  11. 2020IntelligentPooling: Practical Thompson Sampling for mHealthcited by 24
  12. 2021Adapting to Misspecification in Contextual Bandits with Offline Regression Oraclescited by 21
  13. 2018PG-TS: Improved Thompson Sampling for Logistic Contextual Banditscited by 18
  14. 2018The K-Nearest Neighbour UCB algorithm for multi-armed bandits with covariatescited by 16
  15. 2018Multinomial Logit Bandit with Linear Utility Functionscited by 15
  16. 2019Randomized Allocation with Nonparametric Estimation for Contextual Multi-Armed Bandits with Delayed Rewardscited by 15
  17. 2015On the Prior Sensitivity of Thompson Samplingcited by 14
  18. 2020Bayesian Meta-Prior Learning Using Empirical Bayescited by 14
  19. 2020Multitask Bandit Learning Through Heterogeneous Feedback Aggregationcited by 14
  20. 2020Non-Stationary Latent Banditscited by 12
  21. 2020Tractable contextual bandits beyond realizabilitycited by 8
  22. 2017Horde of Bandits using Gaussian Markov Random Fieldscited by 6
  23. 2019Empirical Bayes Regret Minimizationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 9; 2023: 5; 2024: 4; 2025: 3; 2026: 2.

24 papers.

Sort:

By area:

  1. 2021Hierarchical Bayesian Banditscited by 33
  2. 2022Deep Hierarchy in Banditscited by 18
  3. 2022Safe Exploration for Efficient Policy Evaluation and Comparisoncited by 15
  4. 2022Mixed-Effect Thompson Samplingcited by 14
  5. 2023Online Clustering of Bandits with Misspecified User Modelscited by 12
  6. 2022Meta-Learning for Simple Regret Minimizationcited by 10
  7. 2022Towards Scalable and Robust Structured Bandits: A Meta-Learning Frameworkcited by 10
  8. 2022Hierarchical Conversational Preference Elicitation with Bandit Feedbackcited by 10
  9. 2022Multi-Task Off-Policy Learning from Bandit Feedbackcited by 9
  10. 2024Meta Clustering of Neural Banditscited by 7
  11. 2024reBandit: Random Effects based Online RL algorithm for Reducing Cannabis Usecited by 6
  12. 2024Diffusion Models Meet Contextual Banditscited by 5
12 more
  1. 2025Contextual Thompson Sampling via Generation of Missing Datacited by 5
  2. 2022Robust Contextual Linear Banditscited by 4
  3. 2022Thompson Sampling for Robust Transfer in Multi-Task Banditscited by 3
  4. 2023Multiplier Bootstrap-based Explorationcited by 3
  5. 2023Only Pay for What Is Uncertain: Variance-Adaptive Thompson Samplingcited by 3
  6. 2024Multi-Task Combinatorial Bandits for Budget Allocationcited by 2
  7. 2026Human-in-the-Loop Contextual Bandits for Short-Term Rental Dynamic Pricing: Structural Equivalence of Historical Warm-Up and Approval-Gated Live Learningcited by 2
  8. 2023Effect Size Estimation for Duration Recommendation in Online Experiments: Leveraging Hierarchical Models and Objective Utility Approachescited by 0
  9. 2023Zero-Inflated Banditscited by 0
  10. 2025Empirical Bayesian Multi-Bandit Learningcited by 0
  11. 2025Co-Exploration and Co-Exploitation via Shared Structure in Multi-Task Banditscited by 0
  12. 2026Gated Decoupled Compositional Bandits: A Unified Theory of Contextual Bandits with Supervised-Calibrated Action Scaling and Pre-Execution Gatingcited by 0