mlsanityData to 14 Oct 2026

Mixed-Effect Thompson Sampling

arXiv 2205.15124 ↗First posted 30 May 2022cs.LG stat.MLcited by 14

Not on the map. Why?

Abstract (arXiv)

A contextual bandit is a popular framework for online learning to act under uncertainty. In practice, the number of actions is huge and their expected rewards are correlated. In this work, we introduce a general framework for capturing such correlations through a mixed-effect model where actions are related through multiple shared effect parameters. To explore efficiently using this structure, we propose Mixed-Effect Thompson Sampling (meTS) and bound its Bayes regret. The regret bound has two terms, one for learning the action parameters and the other for learning the shared effect parameters. The terms reflect the structure of our model and the quality of priors. Our theoretical findings are validated empirically using both synthetic and real-world problems. We also propose numerous extensions of practical interest. While they do not come with guarantees, they perform well empirically and show the generality of the proposed framework.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 29 of its 58 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

29 of 58 references.

References · 29

  1. 2019Introduction to Multi-Armed Banditscited by 481
  2. 2017A Tutorial on Thompson Samplingcited by 433
  3. 2013Learning to Optimize Via Posterior Samplingcited by 256
  4. 2014An Information-Theoretic Analysis of Thompson Samplingcited by 178
  5. 2020Provable Meta-Learning of Linear Representationscited by 161
  6. 2014Online Clustering of Banditscited by 102
  7. 2021Meta-Thompson Samplingcited by 63
  8. 2020Impact of Representation Learning in Linear Banditscited by 52
  9. 2019Information-Theoretic Confidence Bounds for Reinforcement Learningcited by 51
  10. 2017Multi-Task Learning for Contextual Banditscited by 47
  11. 2020Meta-learning with Stochastic Linear Banditscited by 46
  12. 2021Near-optimal Representation Learning for Linear Bandits and Linear RLcited by 45
17 more
  1. 2020Latent Bandits Revisitedcited by 44
  2. 2021Bayesian decision-making under misspecified priors with applications to meta-learningcited by 44
  3. 2013Thompson Sampling for Complex Bandit Problemscited by 43
  4. 2019Meta Dynamic Pricing: Transfer Learning Across Experimentscited by 42
  5. 2013Sequential Transfer in Multi-armed Bandit with Finite Set of Modelscited by 38
  6. 2017Provably Optimal Algorithms for Generalized Linear Contextual Banditscited by 38
  7. 2021Hierarchical Bayesian Banditscited by 35
  8. 2021No Regrets for Learning the Prior in Banditscited by 30
  9. 2019Top-k Combinatorial Bandits with Full-Bandit Feedbackcited by 28
  10. 2020IntelligentPooling: Practical Thompson Sampling for mHealthcited by 24
  11. 2021Metadata-based Multi-Task Bandits with Bayesian Hierarchical Modelscited by 24
  12. 2022Multi-task Representation Learning with Stochastic Linear Banditscited by 21
  13. 2022Deep Hierarchy in Banditscited by 19
  14. 2021Metalearning Linear Bandits by Prior Updatecited by 16
  15. 2014Bounded Regret for Finite-Armed Structured Banditscited by 15
  16. 2017Variational inference for the multi-armed contextual banditcited by 15
  17. 2022Towards Scalable and Robust Structured Bandits: A Meta-Learning Frameworkcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 7; 2025: 1; 2026: 2.

14 papers.

Sort:

By area:

  1. 2023Exponential Smoothing for Off-Policy Learningcited by 15
  2. 2022Multi-Task Off-Policy Learning from Bandit Feedbackcited by 9
  3. 2024reBandit: Random Effects based Online RL algorithm for Reducing Cannabis Usecited by 6
  4. 2023RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventionscited by 5
  5. 2024Diffusion Models Meet Contextual Banditscited by 5
  6. 2024Bayesian Off-Policy Evaluation and Learning for Large Action Spacescited by 5
  7. 2024Prior-Dependent Allocations for Bayesian Fixed-Budget Best-Arm Identification in Structured Banditscited by 4
  8. 2024A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Liftingcited by 3
  9. 2024Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Samplingcited by 3
  10. 2024Online Posterior Sampling with a Diffusion Priorcited by 2
  11. 2025Empirical Bayesian Multi-Bandit Learningcited by 1
  12. 2023Finite-Time Logarithmic Bayes Regret Upper Boundscited by 0
2 more
  1. 2026Graph Structured Combinatorial Semi-Bandit with Nonlinear Reward Associations through Separable Signalscited by 0
  2. 2026Leveraging Similarities in Multi-Armed Banditscited by 0