mlsanityData to 14 Oct 2026

Stable Thompson Sampling: Valid Inference via Variance Inflation

arXiv 2505.23260 ↗First posted 29 May 2025stat.ML cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

We consider the problem of statistical inference when the data is collected via a Thompson Sampling-type algorithm. While Thompson Sampling (TS) is known to be both asymptotically optimal and empirically effective, its adaptive sampling scheme poses challenges for constructing confidence intervals for model parameters. We propose and analyze a variant of TS, called Stable Thompson Sampling, in which the posterior variance is inflated by a logarithmic factor. We show that this modification leads to asymptotically normal estimates of the arm means, despite the non-i.i.d. nature of the data. Importantly, this statistical benefit comes at a modest cost: the variance inflation increases regret by only a logarithmic factor compared to standard TS. Our results reveal a principled trade-off: by paying a small price in regret, one can enable valid statistical inference for adaptive decision-making algorithms.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 22 of its 59 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

22 of 59 references.

References · 22

  1. 2019Confidence Intervals for Policy Evaluation in Adaptive Experimentscited by 95
  2. 2020Inference for Batched Banditscited by 63
  3. 2013Thompson Sampling for 1-Dimensional Exponential Family Banditscited by 42
  4. 2021Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Banditscited by 40
  5. 2017Accurate Inference for Adaptive Linear Modelscited by 39
  6. 2022Anytime-valid off-policy inference for contextual banditscited by 37
  7. 2017Why Adaptively Collected Data Have Negative Bias and How to Correct for Itcited by 36
  8. 2021Post-Contextual-Bandit Inferencecited by 32
  9. 2021A Closer Look at the Worst-case Behavior of Multi-armed Bandit Algorithmscited by 30
  10. 2019On the bias, risk and consistency of sample means in multi-armed banditscited by 25
  11. 2019Are sample means in multi-armed bandits positively or negatively biased?cited by 24
  12. 2022Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelinescited by 22
10 more
  1. 2021Near-optimal inference in adaptive linear regressioncited by 21
  2. 2024UCB algorithms for multi-armed bandits: Precise regret and adaptive inferencecited by 19
  3. 2019Online Debiasing for Adaptively Collected High-dimensional Data with Applications to Time Series Analysiscited by 13
  4. 2022The Typical Behavior of Bandit Algorithmscited by 12
  5. 2024Inference with the Upper Confidence Bound Algorithmcited by 12
  6. 2023Semi-parametric inference based on adaptively collected datacited by 7
  7. 2023Adaptive Linear Estimating Equationscited by 6
  8. 2023Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inferencecited by 3
  9. 2024Oralytics Reinforcement Learning Algorithmcited by 2
  10. 2015Confidence intervals for AB-testcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2026Bandit Allocational Instabilitycited by 3
  2. 2025Avoiding the Price of Adaptivity: Inference in Linear Contextual Bandits via Stabilitycited by 2
  3. 2026Optimism Stabilizes Thompson Sampling for Adaptive Inferencecited by 2
  4. 2026Efficient Inference after Directionally Stable Adaptive Experimentscited by 1
  5. 2026Bandit Simulation for Average Reward Inferencecited by 0
  6. 2026Statistical Inference for Misspecified Contextual Banditscited by 0
  7. 2026Toward the Optimal Regret-Instability Trade-off in Multi-Armed Banditscited by 0
  8. 2026Sampling Allocation of LinUCB: Optimal Design Limits in the Small-Gap Regimecited by 0