Variance-Aware Feel-Good Thompson Sampling for Contextual Bandits
Not on the map. Why?
Abstract (arXiv)
Variance-dependent regret bounds have received increasing attention in recent studies on contextual bandits. However, most of these studies are focused on upper confidence bound (UCB)-based bandit algorithms, while sampling based bandit algorithms such as Thompson sampling are still understudied. The only exception is the LinVDTS algorithm (Xu et al., 2023), which is limited to linear reward function and its regret bound is not optimal with respect to the model dimension. In this paper, we present FGTSVA, a variance-aware Thompson Sampling algorithm for contextual bandits with general reward function with optimal regret bound. At the core of our analysis is an extension of the decoupling coefficient, a technique commonly used in the analysis of Feel-good Thompson sampling (FGTS) that reflects the complexity of the model space. With the new decoupling coefficient denoted by dc, FGTS-VA achieves the regret of tildeO(√dc·log|F|sumt=1Tσt2+dc), where |F| is the size of the model space, T is the total number of rounds, and σt2 is the subgaussian norm of the noise (e.g., variance when the noise is Gaussian) at round t. In the setting of contextual linear bandits, the regret bound of FGTSVA matches that of UCB-based algorithms using weighted linear regression (Zhou and Gu, 2022).
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 40 references that are in the map. First posted on arXiv 3 Nov 2025.
Builds on
21 of 40 references.
References · 21
- 2020Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processescited by 200
- 2016Why is Posterior Sampling Better than Optimism for Reinforcement Learning?cited by 117
- 2018More Adaptive Algorithms for Adversarial Banditscited by 114
- 2021Feel-Good Thompson Sampling for Contextual Bandits and Reinforcement Learningcited by 66
- 2022Computationally Efficient Horizon-Free Reinforcement Learning for Linear Mixture MDPscited by 52
- 2021Improved Variance-Aware Confidence Sets for Linear Bandits and Linear Mixture MDPcited by 45
- 2021Improved Regret Analysis for Variance-Adaptive Linear Bandits and Horizon-Free Linear Mixture MDPscited by 44
- 2022VOQL: Towards Optimal Regret in Model-free RL with Nonlinear Function Approximationcited by 44
- 2023Variance-Dependent Regret Bounds for Linear Bandits and Reinforcement Learning: Adaptivity and Computational Efficiencycited by 40
- 2022Model-based RL with Optimistic Posterior Sampling: Structural Conditions and Sample Complexitycited by 32
- 2022A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learningcited by 31
- 2020MOTS: Minimax Optimal Thompson Samplingcited by 28
9 more
- 2024Feel-Good Thompson Sampling for Contextual Dueling Banditscited by 27
- 2024More Benefits of Being Distributional: Second-Order Bounds for Reinforcement Learningcited by 23
- 2023Variance-Aware Regret Bounds for Stochastic Contextual Dueling Banditscited by 22
- 2017Efficient-UCBV: An Almost Optimal Algorithm using Variance Estimatescited by 20
- 2020Taking a hint: How to leverage loss predictors in contextual bandits?cited by 20
- 2023Best-of-Three-Worlds Linear Bandit Algorithm with Variance-Adaptive Regret Boundscited by 14
- 2024Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Boundscited by 11
- 2024How Does Variance Shape the Regret in Contextual Bandits?cited by 9
- 2023Only Pay for What Is Uncertain: Variance-Adaptive Thompson Samplingcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.