mlsanityData to 7 Oct 2026

Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback

arXiv 1901.00301 ↗First posted 2 Jan 2019cs.LG stat.MLcited by 21

Not on the map. Why?

Abstract (arXiv)

We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible and helpful in practice.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 2; 2024: 6; 2025: 6; 2026: 4.

21 papers.

Sort:

By area:

  1. 2024LLMs Are In-Context Bandit Reinforcement Learnerscited by 27
  2. 2023Leveraging Demonstrations to Improve Online Learning: Quality Matterscited by 13
  3. 2025Multi-Armed Bandits With Machine Learning-Generated Surrogate Rewardscited by 13
  4. 2022Artificial Replay: A Meta-Algorithm for Harnessing Historical Data in Banditscited by 12
  5. 2024Jump Starting Bandits with LLM-Generated Prior Knowledgecited by 11
  6. 2024Online Bandit Learning with Offline Preference Data for Improved RLHFcited by 6
  7. 2025Best Arm Identification with Possibly Biased Offline Datacited by 4
  8. 2022Thompson Sampling for Robust Transfer in Multi-Task Banditscited by 3
  9. 2025Augmenting Online RL with Offline Data is All You Need: A Unified Hybrid RL Algorithm Design and Analysiscited by 3
  10. 2025Contextual Online Pricing with (Biased) Offline Datacited by 3
  11. 2024Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloningcited by 1
  12. 2026Direction-Aware Offline-to-Online Learning in Linear Contextual Banditscited by 1
9 more
  1. 2021Bandits with Stochastic Experts: Constant Regret, Empirical Experts and Episodescited by 0
  2. 2023Leveraging User-Triggered Supervision in Contextual Banditscited by 0
  3. 2024Identifiable Latent Bandits: Leveraging observational data for personalized decision-makingcited by 0
  4. 2024MBExplainer: Multilevel bandit-based explanations for downstream models with augmented graph embeddingscited by 0
  5. 2025Deconfounded Warm-Start Thompson Sampling with Applications to Precision Medicinecited by 0
  6. 2025Offline Clustering of Preference Learning with Active-data Augmentationcited by 0
  7. 2026Jump Start or False Start? A Theoretical and Empirical Evaluation of LLM-initialized Banditscited by 0
  8. 2026Sample-Mean Anchored Thompson Sampling for Offline-to-Online Learning with Distribution Shiftcited by 0
  9. 2026Personalizing Large Language Model Agents with Small Policy Modelscited by 0