mlsanityData to 7 Oct 2026

Practical Contextual Bandits with Regression Oracles

arXiv 1803.01088 ↗First posted 3 Mar 2018cs.LG stat.MLcited by 92

Not on the map. Why?

Abstract (arXiv)

A major challenge in contextual bandits is to design general-purpose algorithms that are both practically useful and theoretically well-founded. We present a new technique that has the empirical and computational advantages of realizability-based approaches combined with the flexibility of agnostic methods. Our algorithms leverage the availability of a regression oracle for the value-function class, a more realistic and reasonable oracle than the classification oracles over policies typically assumed by agnostic methods. Our approach generalizes both UCB and LinUCB to far more expressive possible model classes and achieves low regret under certain distributional assumptions. In an extensive empirical evaluation, compared to both realizability-based and agnostic baselines, we find that our approach typically gives comparable or superior results.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 14; 2023: 18; 2024: 17; 2025: 11; 2026: 16.

92 papers.

Sort:

By area:

  1. 2021Adapting to Misspecification in Contextual Banditscited by 81
  2. 2021Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximationcited by 69
  3. 2021Efficient First-Order Contextual Bandits: Prediction, Allocation, and Triangular Discriminationcited by 53
  4. 2023Oracle Efficient Online Multicalibration and Omnipredictioncited by 36
  5. 2021Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvaturecited by 32
  6. 2022Contextual Bandits with Large Action Spaces: Made Practicalcited by 32
  7. 2023Multicalibration as Boosting for Regressioncited by 32
  8. 2022Contextual Bandits with Packing and Covering Constraints: A Modular Lagrangian Approach via Regressioncited by 30
  9. 2023Improved Regret Bounds of (Multinomial) Logistic Bandits via Regret-to-Confidence-Set Conversioncited by 24
  10. 2024More Benefits of Being Distributional: Second-Order Bounds for Reinforcement Learningcited by 23
  11. 2021Adapting to Misspecification in Contextual Bandits with Offline Regression Oraclescited by 21
  12. 2022Efficient Active Learning with Abstentioncited by 21
38 more
  1. 2021Design of Experiments for Stochastic Contextual Linear Banditscited by 18
  2. 2023Bypassing the Simulator: Near-Optimal Adversarial Linear Contextual Banditscited by 18
  3. 2022Contextual Bandits with Smooth Regret: Efficient Learning in Continuous Action Spacescited by 17
  4. 2022Instance-optimal PAC Algorithms for Contextual Banditscited by 16
  5. 2024Efficient Sequential Decision Making with Large Language Modelscited by 16
  6. 2021On component interactions in two-stage recommender systemscited by 14
  7. 2023Oracle-Efficient Pessimism: Offline Policy Optimization in Contextual Banditscited by 14
  8. 2021Provably Correct Optimization and Exploration with Non-linear Policiescited by 13
  9. 2022Active Learning with Neural Networks: Insights from Nonparametric Statisticscited by 13
  10. 2022Optimal Contextual Bandits with Knapsacks under Realizability via Regression Oraclescited by 12
  11. 2022Optimism in Face of a Context: Regret Guarantees for Stochastic Contextual MDPcited by 11
  12. 2024PAK-UCB Contextual Bandit: An Online Learning Approach to Prompt-Aware Selection of Generative Models and LLMscited by 11
  13. 2023Practical Contextual Bandits with Feedback Graphscited by 10
  14. 2023Tracking Most Significant Shifts in Nonparametric Contextual Banditscited by 10
  15. 2023Best-of-Both-Worlds Algorithms for Linear Contextual Banditscited by 10
  16. 2021Going Beyond Linear RL: Sample Efficient Neural Function Approximationcited by 9
  17. 2022Eluder-based Regret for Stochastic Contextual MDPscited by 9
  18. 2023Efficient Rate Optimal Regret for Adversarial Contextual MDPs Using Online Function Approximationcited by 9
  19. 2023Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learningcited by 9
  20. 2023Pearl: A Production-ready Reinforcement Learning Agentcited by 9
  21. 2021Top-k eXtreme Contextual Bandits with Arm Hierarchycited by 8
  22. 2023LLMs-augmented Contextual Banditcited by 8
  23. 2024The Central Role of the Loss Function in Reinforcement Learningcited by 8
  24. 2022Adaptive Oracle-Efficient Online Learningcited by 7
  25. 2022Global Optimization with Parametric Function Approximationcited by 7
  26. 2024The Real Price of Bandit Information in Multiclass Classificationcited by 7
  27. 2024Offline Oracle-Efficient Learning for Contextual MDPs via Layerwise Exploration-Exploitation Tradeoffcited by 7
  28. 2025Large Language Model-Enhanced Multi-Armed Banditscited by 7
  29. 2025Contextual Online Uncertainty-Aware Preference Learning for Human Feedbackcited by 7
  30. 2024No-Regret is not enough! Bandits with General Constraints through Adaptive Regret Minimizationcited by 6
  31. 2025Single Index Bandits: Generalized Linear Contextual Bandits with Unknown Reward Functionscited by 6
  32. 2024Experiment Planning with Function Approximationcited by 5
  33. 2021Parallelizing Contextual Banditscited by 4
  34. 2021Towards Costless Model Selection in Contextual Bandits: A Bias-Variance Perspectivecited by 4
  35. 2021Dealing with Expert Bias in Collective Decision-Makingcited by 4
  36. 2022Context Uncertainty in Contextual Bandits with Applications to Recommender Systemscited by 4
  37. 2023Provably Efficient Reinforcement Learning via Surprise Boundcited by 4
  38. 2023Online Learning in Contextual Second-Price Pay-Per-Click Auctionscited by 4

50 most cited of 92 All 92 (JSON)