mlsanityData to 7 Oct 2026

On Kernelized Multi-armed Bandits

arXiv 1704.00445 ↗First posted 3 Apr 2017cs.LGcited by 348

Not on the map. Why?

Abstract (arXiv)

We consider the stochastic bandit problem with a continuous set of arms, with the expected reward function over the arms assumed to be fixed but unknown. We provide two new Gaussian process-based algorithms for continuous bandit optimization-Improved GP-UCB (IGP-UCB) and GP-Thomson sampling (GP-TS), and derive corresponding regret bounds. Specifically, the bounds hold when the expected reward function belongs to the reproducing kernel Hilbert space (RKHS) that naturally corresponds to a Gaussian process kernel used as input by the algorithms. Along the way, we derive a new self-normalized concentration inequality for vector- valued martingales of arbitrary, possibly infinite, dimension. Finally, experimental evaluation and comparisons to existing algorithms on synthetic and real-world environments are carried out that highlight the favorable gains of the proposed strategies in many cases.

Key facts. Cited by 348 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 39; 2022: 53; 2023: 61; 2024: 67; 2025: 61; 2026: 66.

348 papers.

Sort:

By area:

  1. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 409
  2. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  3. 2023Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimismcited by 69
  4. 2021Gaussian Process Bandit Optimization with Few Batchescited by 57
  5. 2021Optimal Order Simple Regret for Gaussian Process Banditscited by 52
  6. 2021Misspecified Gaussian Process Bandit Optimizationcited by 52
  7. 2021Practical and Rigorous Uncertainty Bounds for Gaussian Process Regressioncited by 49
  8. 2023State-wise Safe Reinforcement Learning: A Surveycited by 48
  9. 2021Neural Contextual Bandits without Regretcited by 40
  10. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  11. 2022Robust Multi-Objective Bayesian Optimization Under Input Noisecited by 36
  12. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
38 more
  1. 2021Differentially Private Federated Bayesian Optimization with Distributed Explorationcited by 32
  2. 2023Optimistic Active Exploration of Dynamical Systemscited by 32
  3. 2021Weighted Gaussian Process Bandits for Non-stationary Environmentscited by 30
  4. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  5. 2021GoSafe: Globally Optimal Safe Robot Learningcited by 27
  6. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 27
  7. 2024Checkpoint Merging via Bayesian Optimization in LLM Pretrainingcited by 27
  8. 2024A Unified Confidence Sequence for Generalized Linear Models, with Applications to Banditscited by 27
  9. 2022Sample-Then-Optimize Batch Neural Thompson Samplingcited by 25
  10. 2023On the Sublinear Regret of GP-UCBcited by 25
  11. 2021Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximationcited by 23
  12. 2022On Kernelized Multi-Armed Bandits with Constraintscited by 23
  13. 2021Neural Active Learning with Performance Guaranteescited by 22
  14. 2021Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimizationcited by 22
  15. 2022Surrogate modeling for Bayesian optimization beyond a single Gaussian processcited by 22
  16. 2023Efficient Exploration in Continuous-time Model-based Reinforcement Learningcited by 21
  17. 2021On Reward-Free RL with Kernel and Neural Function Approximations: Single-Agent MDP and Markov Gamecited by 20
  18. 2023Kernelized Reinforcement Learning with Order Optimal Regret Boundscited by 20
  19. 2021No-Regret Algorithms for Time-Varying Bayesian Optimizationcited by 19
  20. 2021Value-at-Risk Optimization with Gaussian Processescited by 19
  21. 2022Improved Convergence Rates for Sparse Approximation Methods in Kernel-Based Learningcited by 18
  22. 2023Sample Efficient Preference Alignment in LLMs via Active Explorationcited by 18
  23. 2021Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applicationscited by 17
  24. 2021Nearly Optimal Algorithms for Level Set Estimationcited by 17
  25. 2021Risk-averse Heteroscedastic Bayesian Optimizationcited by 17
  26. 2022GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systemscited by 17
  27. 2022Learning Neural Contextual Bandits Through Perturbed Rewardscited by 17
  28. 2022Federated Neural Banditscited by 17
  29. 2022An Optimization-based Algorithm for Non-stationary Kernel Bandits without Prior Knowledgecited by 17
  30. 2023Randomized Gaussian Process Upper Confidence Bound with Tighter Bayesian Regret Boundscited by 16
  31. 2023Random Exploration in Bayesian Optimization: Order-Optimal Regret and Computational Efficiencycited by 16
  32. 2024Principled Preferential Bayesian Optimizationcited by 16
  33. 2024Cost-aware Bayesian Optimization via the Pandora's Box Gittins Indexcited by 16
  34. 2022Communication Efficient Distributed Learning for Kernelized Contextual Banditscited by 15
  35. 2023Posterior Sampling-Based Bayesian Optimization with Tighter Bayesian Regret Boundscited by 15
  36. 2021Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learningcited by 14
  37. 2021Robust Uncertainty Bounds in Reproducing Kernel Hilbert Spaces: A Convex Optimization Approachcited by 14
  38. 2022Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computationcited by 14

50 most cited of 348 All 348 (JSON)