mlsanityData to 7 Oct 2026

Inference with the Upper Confidence Bound Algorithm

arXiv 2408.04595 ↗First posted 8 Aug 2024stat.ML cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we discuss the asymptotic behavior of the Upper Confidence Bound (UCB) algorithm in the context of multiarmed bandit problems and discuss its implication in downstream inferential tasks. While inferential tasks become challenging when data is collected in a sequential manner, we argue that this problem can be alleviated when the sequential algorithm at hand satisfies certain stability property. This notion of stability is motivated from the seminal work of Lai and Wei (1982). Our first main result shows that such a stability property is always satisfied for the UCB algorithm, and as a result the sample means for each arm are asymptotically normal. Next, we examine the stability properties of the UCB algorithm when the number of arms K is allowed to grow with the number of arm pulls T. We show that in such a case the arms are stable when fraclog Klog T rightarrow 0, and the number of near-optimal arms are large.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 12 of its 33 references that are in the map. First posted on arXiv 8 Aug 2024.

Builds on

12 of 33 references.

References · 12

  1. 2019Confidence Intervals for Policy Evaluation in Adaptive Experimentscited by 95
  2. 2021Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Banditscited by 40
  3. 2017Accurate Inference for Adaptive Linear Modelscited by 39
  4. 2022Anytime-valid off-policy inference for contextual banditscited by 37
  5. 2021Post-Contextual-Bandit Inferencecited by 32
  6. 2021A Closer Look at the Worst-case Behavior of Multi-armed Bandit Algorithmscited by 29
  7. 2019On the bias, risk and consistency of sample means in multi-armed banditscited by 24
  8. 2021Near-optimal inference in adaptive linear regressioncited by 20
  9. 2019Online Debiasing for Adaptively Collected High-dimensional Data with Applications to Time Series Analysiscited by 13
  10. 2023Semi-parametric inference based on adaptively collected datacited by 6
  11. 2023Adaptive Linear Estimating Equationscited by 6
  12. 2023Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inferencecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 5.

9 papers.

  1. 2025Stable Thompson Sampling: Valid Inference via Variance Inflationcited by 6
  2. 2024Allocation Stability and Wald Inference under Variance-Aware UCBcited by 4
  3. 2025Simulation-Based Inference for Adaptive Experimentscited by 3
  4. 2025Avoiding the Price of Adaptivity: Inference in Linear Contextual Bandits via Stabilitycited by 2
  5. 2026Optimism Stabilizes Thompson Sampling for Adaptive Inferencecited by 2
  6. 2026Bandit Allocational Instabilitycited by 2
  7. 2026Efficient Inference after Directionally Stable Adaptive Experimentscited by 1
  8. 2026Bandit Simulation for Average Reward Inferencecited by 0
  9. 2026Toward the Optimal Regret-Instability Trade-off in Multi-Armed Banditscited by 0