# Asymptotically Optimal Problem-Dependent Bandit Policies for Transfer Learning

arXiv: https://arxiv.org/abs/2509.19098 · HTML: https://mlsanity.com/paper/2509.19098/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Builds on (most cited first, 6 of 6)

- [Sequential Transfer in Multi-armed Bandit with Finite Set of Models](https://mlsanity.com/paper/1307.6887/) (2013)
- [Transfer Learning for Contextual Multi-armed Bandits](https://mlsanity.com/paper/2211.12612/) (2022)
- [Transfer Learning in Latent Contextual Bandits with Covariate Shift Through Causal Transportability](https://mlsanity.com/paper/2502.20153/) (2025)
- [Thompson Sampling for Robust Transfer in Multi-Task Bandits](https://mlsanity.com/paper/2206.08556/) (2022)
- [Transfer in Sequential Multi-armed Bandits via Reward Samples](https://mlsanity.com/paper/2403.12428/) (2024)
- [Exploiting Adjacent Similarity in Multi-Armed Bandit Tasks via Transfer of Reward Samples](https://mlsanity.com/paper/2409.19975/) (2024)

## Abstract (arXiv)

We study the non-contextual multi-armed bandit problem in a transfer learning setting: before any pulls, the learner is given N'_k i.i.d. samples from each source distribution nu'_k, and the true target distributions nu_k lie within a known distance bound d_k(nu_k, nu'_k) <= L_k. In this framework, we first derive a problem-dependent asymptotic lower bound on cumulative regret that extends the classical Lai-Robbins result to incorporate the transfer parameters (d_k, L_k, N'_k). We then propose KL-UCB-Transfer, a simple index policy that matches this new bound in the Gaussian case. Finally, we validate our approach via simulations, showing that KL-UCB-Transfer significantly outperforms the no-prior baseline when source and target distributions are sufficiently close.