# Offline-to-online hyperparameter transfer for stochastic bandits arXiv: https://arxiv.org/abs/2501.02926 ยท HTML: https://mlsanity.com/paper/2501.02926/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. - Builds on 34 of its 64 references that are in the map. - First posted on arXiv 6 Jan 2025. ## Placed in - Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/) ## Builds on (most cited first, 30 of 34) - [Adaptive Gradient-Based Meta-Learning Methods](https://mlsanity.com/paper/1906.02717/) (2019) - [Online Meta-Learning](https://mlsanity.com/paper/1902.08438/) (2019) - [Tsallis-INF: An Optimal Algorithm for Stochastic and Adversarial Bandits](https://mlsanity.com/paper/1807.07623/) (2018) - [Corralling a Band of Bandit Algorithms](https://mlsanity.com/paper/1612.06246/) (2016) - [Adapting to Misspecification in Contextual Bandits](https://mlsanity.com/paper/2107.05745/) (2021) - [Model Selection in Contextual Stochastic Bandit Problems](https://mlsanity.com/paper/2003.01704/) (2020) - [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021) - [Model selection for contextual bandits](https://mlsanity.com/paper/1906.00531/) (2019) - [A PAC Approach to Application-Specific Algorithm Selection](https://mlsanity.com/paper/1511.07147/) (2015) - [Field Study in Deploying Restless Multi-Armed Bandits: Assisting Non-Profits in Improving Maternal and Child Health](https://mlsanity.com/paper/2109.08075/) (2021) - [Sequential Transfer in Multi-armed Bandit with Finite Set of Models](https://mlsanity.com/paper/1307.6887/) (2013) - [Dispersion for Data-Driven Algorithm Design, Online Learning, and Private Optimization](https://mlsanity.com/paper/1711.03091/) (2017) - [Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior](https://mlsanity.com/paper/1811.09558/) (2018) - [Learning-Theoretic Foundations of Algorithm Configuration for Combinatorial Partitioning Problems](https://mlsanity.com/paper/1611.04535/) (2016) - [Population-Based Black-Box Optimization for Biological Sequence Design](https://mlsanity.com/paper/2006.03227/) (2020) - [The Pareto Frontier of model selection for general Contextual Bandits](https://mlsanity.com/paper/2110.13282/) (2021) - [Generalization Bounds for Data-Driven Numerical Linear Algebra](https://mlsanity.com/paper/2206.07886/) (2022) - [Corralling Stochastic Bandit Algorithms](https://mlsanity.com/paper/2006.09255/) (2020) - [Corralling a Larger Band of Bandits: A Case Study on Switching Regret for Linear Bandits](https://mlsanity.com/paper/2202.06151/) (2022) - [Efficient-UCBV: An Almost Optimal Algorithm using Variance Estimates](https://mlsanity.com/paper/1711.03591/) (2017) - [Data-Driven Clustering via Parameterized Lloyd's Families](https://mlsanity.com/paper/1809.06987/) (2018) - [Metalearning Linear Bandits by Prior Update](https://mlsanity.com/paper/2107.05320/) (2021) - [Distributionally Robust Bayesian Optimization with varphi-divergences](https://mlsanity.com/paper/2203.02128/) (2022) - [Data driven semi-supervised learning](https://mlsanity.com/paper/2103.10547/) (2021) - [Provably tuning the ElasticNet across instances](https://mlsanity.com/paper/2207.10199/) (2022) - [Learning Complexity of Simulated Annealing](https://mlsanity.com/paper/2003.02981/) (2020) - [An Analysis of Robustness of Non-Lipschitz Networks](https://mlsanity.com/paper/2010.06154/) (2020) - [Best of Both Worlds Model Selection](https://mlsanity.com/paper/2206.14912/) (2022) - [Syndicated Bandits: A Framework for Auto Tuning Hyper-parameters in Contextual Bandit Algorithms](https://mlsanity.com/paper/2106.02979/) (2021) - [Meta-Learning for Simple Regret Minimization](https://mlsanity.com/paper/2202.12888/) (2022) ## Built on by (most cited first, 7 of 7) - [KABB: Knowledge-Aware Bayesian Bandits for Dynamic Expert Coordination in Multi-Agent Systems](https://mlsanity.com/paper/2502.07350/) (2025) - [Conservative classifiers do consistently well with improving agents: characterizing statistical and online learning](https://mlsanity.com/paper/2506.05252/) (2025) - [Distribution-dependent Generalization Bounds for Tuning Linear Regression Across Tasks](https://mlsanity.com/paper/2507.05084/) (2025) - [Algorithm Design and Stronger Guarantees for the Improving Multi-Armed Bandits Problem](https://mlsanity.com/paper/2511.10619/) (2025) - [Tuning Algorithmic and Architectural Hyperparameters in Graph-Based Semi-Supervised Learning with Provable Guarantees](https://mlsanity.com/paper/2502.12937/) (2025) - [Gradient Descent with Provably Tuned Learning-rate Schedules](https://mlsanity.com/paper/2512.05084/) (2025) - [Algorithmic Approaches to Sequential Decision-Making and Social Epistemology](https://mlsanity.com/paper/2607.20636/) (2026) ## Abstract (arXiv) Classic algorithms for stochastic bandits typically use hyperparameters that govern their critical properties such as the trade-off between exploration and exploitation. Tuning these hyperparameters is a problem of great practical significance. However, this is a challenging problem and in certain cases is information theoretically impossible. To address this challenge, we consider a practically relevant transfer learning setting where one has access to offline data collected from several bandit problems (tasks) coming from an unknown distribution over the tasks. Our aim is to use this offline data to set the hyperparameters for a new task drawn from the unknown distribution. We provide bounds on the inter-task (number of tasks) and intra-task (number of arm pulls for each task) sample complexity for learning near-optimal hyperparameters on unseen tasks drawn from the distribution. Our results apply to several classic algorithms, including tuning the exploration parameters in UCB and LinUCB and the noise parameter in GP-UCB. Our experiments indicate the significance and effectiveness of the transfer of hyperparameters from offline problems in online learning with stochastic bandit feedback.