mlsanityData to 14 Oct 2026

Offline-to-online hyperparameter transfer for stochastic bandits

arXiv 2501.02926 ↗First posted 6 Jan 2025cs.LGcited by 7

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Classic algorithms for stochastic bandits typically use hyperparameters that govern their critical properties such as the trade-off between exploration and exploitation. Tuning these hyperparameters is a problem of great practical significance. However, this is a challenging problem and in certain cases is information theoretically impossible. To address this challenge, we consider a practically relevant transfer learning setting where one has access to offline data collected from several bandit problems (tasks) coming from an unknown distribution over the tasks. Our aim is to use this offline data to set the hyperparameters for a new task drawn from the unknown distribution. We provide bounds on the inter-task (number of tasks) and intra-task (number of arm pulls for each task) sample complexity for learning near-optimal hyperparameters on unseen tasks drawn from the distribution. Our results apply to several classic algorithms, including tuning the exploration parameters in UCB and LinUCB and the noise parameter in GP-UCB. Our experiments indicate the significance and effectiveness of the transfer of hyperparameters from offline problems in online learning with stochastic bandit feedback.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 34 of its 64 references that are in the map. First posted on arXiv 6 Jan 2025.

Builds on

34 of 64 references.

References · 34

  1. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  2. 2019Online Meta-Learningcited by 137
  3. 2018Tsallis-INF: An Optimal Algorithm for Stochastic and Adversarial Banditscited by 124
  4. 2016Corralling a Band of Bandit Algorithmscited by 100
  5. 2021Adapting to Misspecification in Contextual Banditscited by 81
  6. 2020Model Selection in Contextual Stochastic Bandit Problemscited by 71
  7. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  8. 2019Model selection for contextual banditscited by 60
  9. 2015A PAC Approach to Application-Specific Algorithm Selectioncited by 58
  10. 2021Field Study in Deploying Restless Multi-Armed Bandits: Assisting Non-Profits in Improving Maternal and Child Healthcited by 43
  11. 2013Sequential Transfer in Multi-armed Bandit with Finite Set of Modelscited by 38
  12. 2017Dispersion for Data-Driven Algorithm Design, Online Learning, and Private Optimizationcited by 37
22 more
  1. 2018Regret bounds for meta Bayesian optimization with an unknown Gaussian process priorcited by 31
  2. 2016Learning-Theoretic Foundations of Algorithm Configuration for Combinatorial Partitioning Problemscited by 30
  3. 2020Population-Based Black-Box Optimization for Biological Sequence Designcited by 29
  4. 2021The Pareto Frontier of model selection for general Contextual Banditscited by 24
  5. 2022Generalization Bounds for Data-Driven Numerical Linear Algebracited by 24
  6. 2020Corralling Stochastic Bandit Algorithmscited by 23
  7. 2022Corralling a Larger Band of Bandits: A Case Study on Switching Regret for Linear Banditscited by 22
  8. 2017Efficient-UCBV: An Almost Optimal Algorithm using Variance Estimatescited by 20
  9. 2018Data-Driven Clustering via Parameterized Lloyd's Familiescited by 17
  10. 2021Metalearning Linear Bandits by Prior Updatecited by 16
  11. 2022Distributionally Robust Bayesian Optimization with varphi-divergencescited by 16
  12. 2021Data driven semi-supervised learningcited by 14
  13. 2022Provably tuning the ElasticNet across instancescited by 14
  14. 2020Learning Complexity of Simulated Annealingcited by 12
  15. 2020An Analysis of Robustness of Non-Lipschitz Networkscited by 12
  16. 2022Best of Both Worlds Model Selectioncited by 11
  17. 2021Syndicated Bandits: A Framework for Auto Tuning Hyper-parameters in Contextual Bandit Algorithmscited by 10
  18. 2022Meta-Learning for Simple Regret Minimizationcited by 10
  19. 2023Wasserstein Distributionally Robust Policy Evaluation and Learning for Contextual Banditscited by 10
  20. 2023Meta-Learning Adversarial Bandit Algorithmscited by 8
  21. 2020Upper Confidence Bounds for Combining Stochastic Banditscited by 7
  22. 2024Learning accurate and interpretable tree-based modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 1.

7 papers.

  1. 2025KABB: Knowledge-Aware Bayesian Bandits for Dynamic Expert Coordination in Multi-Agent Systemscited by 37
  2. 2025Conservative classifiers do consistently well with improving agents: characterizing statistical and online learningcited by 2
  3. 2025Distribution-dependent Generalization Bounds for Tuning Linear Regression Across Taskscited by 2
  4. 2025Algorithm Design and Stronger Guarantees for the Improving Multi-Armed Bandits Problemcited by 2
  5. 2025Tuning Algorithmic and Architectural Hyperparameters in Graph-Based Semi-Supervised Learning with Provable Guaranteescited by 1
  6. 2025Gradient Descent with Provably Tuned Learning-rate Schedulescited by 1
  7. 2026Algorithmic Approaches to Sequential Decision-Making and Social Epistemologycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Offline-to-Online Learning in Linear Bandits
  2. 2026Offline Local Search for Online Stochastic Bandits
  3. 2021Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization
  4. 2026Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits
  5. 2024Is Offline Decision Making Possible with Only Few Samples? Reliable Decisions in Data-Starved Bandits via Trust Region Enhancement
  6. 2026Bandits in Prod: Hyperparameter Optimization at Inference Time
  7. 2026Gaussian-Width Bounds for Offline-to-Online Linear Bandits
  8. 2020Hyperparameter Selection for Offline Reinforcement Learning