# Smooth Bandit Optimization: Generalization to H"older Space

arXiv: https://arxiv.org/abs/2012.06076 · HTML: https://mlsanity.com/paper/2012.06076/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2020.

## Built on by (most cited first, 10 of 10)

- [No-Regret Reinforcement Learning in Smooth MDPs](https://mlsanity.com/paper/2402.03792/) (2024)
- [Instance-Dependent Regret Analysis of Kernelized Bandits](https://mlsanity.com/paper/2203.06297/) (2022)
- [Adaptation to Misspecified Kernel Regularity in Kernelised Bandits](https://mlsanity.com/paper/2304.13830/) (2023)
- [Low-Rank MDPs with Continuous Action Spaces](https://mlsanity.com/paper/2311.03564/) (2023)
- [Local Linearity: the Key for No-regret Reinforcement Learning in Continuous MDPs](https://mlsanity.com/paper/2410.24071/) (2024)
- [Consequences of Kernel Regularity for Bandit Optimization](https://mlsanity.com/paper/2512.05957/) (2025)
- [Smoothness-Adaptive Dynamic Pricing with Nonparametric Demand Learning](https://mlsanity.com/paper/2310.07558/) (2023)
- [Towards a Theoretical Understanding of Two Tower Recommendation Models](https://mlsanity.com/paper/2403.00802/) (2024)
- [Finite Sample Bounds for Non-Parametric Regression: Optimal Sample Efficiency and Space Complexity](https://mlsanity.com/paper/2412.14744/) (2024)
- [Online Market Making and the Value of Observing the Order Book](https://mlsanity.com/paper/2605.19584/) (2026)

## Abstract (arXiv)

We consider bandit optimization of a smooth reward function, where the goal is cumulative regret minimization. This problem has been studied for α-H"older continuous (including Lipschitz) functions with 0<α≤ 1. Our main result is in generalization of the reward function to H"older space with exponent α>1 to bridge the gap between Lipschitz bandits and infinitely-differentiable models such as linear bandits. For H"older continuous functions, approaches based on random sampling in bins of a discretized domain suffices as optimal. In contrast, we propose a class of two-layer algorithms that deploy misspecified linear/polynomial bandit algorithms in bins. We demonstrate that the proposed algorithm can exploit higher-order smoothness of the function by deriving a regret upper bound of tildeO(T^fracd+αd+2α) for when α>1, which matches existing lower bound. We also study adaptation to unknown function smoothness over a continuous scale of H"older spaces indexed by α, with a bandit model selection approach applied with our proposed two-layer algorithms. We show that it achieves regret rate that matches the existing lower bound for adaptation within the α≤ 1 subset.