mlsanityData to 14 Oct 2026

Transfer Learning for Contextual Multi-armed Bandits

arXiv 2211.12612 ↗First posted 22 Nov 2022stat.ML cs.LG math.STcited by 24

Not on the map. Why?

Abstract (arXiv)

Motivated by a range of applications, we study in this paper the problem of transfer learning for nonparametric contextual multi-armed bandits under the covariate shift model, where we have data collected on source bandits before the start of the target bandit learning. The minimax rate of convergence for the cumulative regret is established and a novel transfer learning algorithm that attains the minimax regret is proposed. The results quantify the contribution of the data from the source domains for learning in the target domain in the context of nonparametric contextual multi-armed bandits. In view of the general impossibility of adaptation to unknown smoothness, we develop a data-driven algorithm that achieves near-optimal statistical guarantees (up to a logarithmic factor) while automatically adapting to the unknown parameters over a large collection of parameter spaces under an additional self-similarity assumption. A simulation study is carried out to illustrate the benefits of utilizing the data from the auxiliary source domains for learning in the target domain.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 25 of its 95 references that are in the map. First posted on arXiv 22 Nov 2022.

Builds on

25 of 95 references.

References · 25

  1. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 368
  2. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  3. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 287
  4. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 279
  5. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 161
  6. 2020Batch Value-function Approximation with Only Realizabilitycited by 104
  7. 2020Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimalitycited by 90
  8. 2017Mostly Exploration-Free Algorithms for Contextual Banditscited by 85
  9. 2018Marginal Singularity, and the Benefits of Labels in Covariate-Shiftcited by 65
  10. 2019Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifiercited by 65
  11. 2020On the Value of Target Data in Transfer Learningcited by 65
  12. 2022Optimally tackling covariate shift in RKHS-based nonparametric regressioncited by 54
13 more
  1. 2017MNL-Bandit: A Dynamic Learning Approach to Assortment Selectioncited by 50
  2. 2022Leveraging Offline Data in Online Reinforcement Learningcited by 42
  3. 2020Transfer Learning in Large-scale Gaussian Graphical Models with False Discovery Rate Controlcited by 38
  4. 2022A new similarity measure for covariate shift with applications to nonparametric regressioncited by 33
  5. 2019Smooth Contextual Bandits: Bridging the Parametric and Non-differentiable Regret Regimescited by 25
  6. 2020Minimax optimal approaches to the label shift problem in non-parametric settingscited by 20
  7. 2018The K-Nearest Neighbour UCB algorithm for multi-armed bandits with covariatescited by 16
  8. 2019Smoothness-Adaptive Contextual Banditscited by 15
  9. 2016Dynamic Assortment Personalization in High Dimensionscited by 14
  10. 2020Self-Tuning Bandits over Unknown Covariate-Shiftscited by 12
  11. 2021Bandit Algorithms for Precision Medicinecited by 12
  12. 2021ESCADA: Efficient Safety and Context Aware Dose Allocation for Precision Medicinecited by 5
  13. 2019A Biologically Plausible Benchmark for Contextual Bandit Algorithms in Precision Oncology Using in vitro Datacited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 8; 2025: 9; 2026: 5.

24 papers.

Sort:

By area:

  1. 2023Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learningcited by 16
  2. 2023Tracking Most Significant Shifts in Nonparametric Contextual Banditscited by 10
  3. 2024Contextual Bandits for Unbounded Context Distributionscited by 8
  4. 2024Adaptive Smooth Non-Stationary Banditscited by 4
  5. 2025Transfer Learning in Latent Contextual Bandits with Covariate Shift Through Causal Transportabilitycited by 4
  6. 2024Batched Nonparametric Contextual Banditscited by 3
  7. 2024Trans-Glasso: A Transfer Learning Approach to Precision Matrix Estimationcited by 2
  8. 2025Transfer Faster, Price Smarter: Minimax Dynamic Pricing under Cross-Market Preference Shiftcited by 2
  9. 2024Transfer Learning for Spatial Autoregressive Models with Application to U.S. Presidential Election Predictioncited by 1
  10. 2024Exploiting Adjacent Similarity in Multi-Armed Bandit Tasks via Transfer of Reward Samplescited by 1
  11. 2025Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Datacited by 1
  12. 2026Direction-Aware Offline-to-Online Learning in Linear Contextual Banditscited by 1
12 more
  1. 2024An adaptive transfer learning perspective on classification in non-stationary environmentscited by 0
  2. 2024Minimax-optimal trust-aware multi-armed banditscited by 0
  3. 2025Parameter-Adaptive Dynamic Pricingcited by 0
  4. 2025Locally Private Nonparametric Contextual Multi-armed Banditscited by 0
  5. 2025Transfer Learning for Matrix Completioncited by 0
  6. 2025Vector preference-based contextual bandits under distributional shiftscited by 0
  7. 2025Asymptotically Optimal Problem-Dependent Bandit Policies for Transfer Learningcited by 0
  8. 2025The Adaptivity Barrier in Batched Nonparametric Bandits: Sharp Characterization of the Price of Unknown Margincited by 0
  9. 2026Jump Start or False Start? A Theoretical and Empirical Evaluation of LLM-initialized Banditscited by 0
  10. 2026Sample Complexity of Transfer Learning: An Optimal Transport Approachcited by 0
  11. 2026Multi-Armed Bandits with Arriving Arms: Sequential Screening, Dynamic Regret, and Sublinear Guaranteescited by 0
  12. 2026Transfer Learning in High-dimensional Ising Modelscited by 0