mlsanityData to 14 Oct 2026

Inference for Batched Bandits

arXiv 2002.03217 ↗First posted 8 Feb 2020cs.LG stat.MLcited by 63

Not on the map. Why?

Abstract (arXiv)

As bandit algorithms are increasingly utilized in scientific studies and industrial applications, there is an associated increasing need for reliable inference methods based on the resulting adaptively-collected data. In this work, we develop methods for inference on data collected in batches using a bandit algorithm. We first prove that the ordinary least squares estimator (OLS), which is asymptotically normal on independently sampled data, is not asymptotically normal on data collected using standard bandit algorithms when there is no unique optimal arm. This asymptotic non-normality result implies that the naive assumption that the OLS estimator is approximately normal can lead to Type-1 error inflation and confidence intervals with below-nominal coverage probabilities. Second, we introduce the Batched OLS estimator (BOLS) that we prove is (1) asymptotically normal on data collected from both multi-arm and contextual bandits and (2) robust to non-stationarity in the baseline reward.

Key facts. Cited by 63 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 7; 2023: 9; 2024: 12; 2025: 13; 2026: 6.

63 papers.

Sort:

By area:

  1. 2021Statistical Inference with M-Estimators on Adaptively Collected Datacited by 40
  2. 2021Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Banditscited by 40
  3. 2022Anytime-valid off-policy inference for contextual banditscited by 37
  4. 2021Deeply-Debiased Off-Policy Interval Estimationcited by 36
  5. 2021A Closer Look at the Worst-case Behavior of Multi-armed Bandit Algorithmscited by 30
  6. 2021Online Multi-Armed Bandits with Adaptive Inferencecited by 22
  7. 2021Near-optimal inference in adaptive linear regressioncited by 21
  8. 2021Lipschitz Bandits with Batched Feedbackcited by 21
  9. 2022Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Carecited by 18
  10. 2021Weak Signal Asymptotics for Sequentially Randomized Experimentscited by 14
  11. 2022Counterfactual inference in sequential experimentscited by 13
  12. 2024Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choicescited by 12
38 more
  1. 2021Efficient Online Estimation of Causal Effects by Deciding What to Observecited by 11
  2. 2023A Lipschitz Bandits Approach for Continuous Hyperparameter Optimizationcited by 9
  3. 2023An Experimental Design for Anytime-Valid Causal Inference on Multi-Armed Banditscited by 9
  4. 2024Online Experimental Design With Estimation-Regret Trade-off Under Network Interferencecited by 9
  5. 2021From Finite to Countable-Armed Banditscited by 8
  6. 2024Linear Contextual Bandits with Interferencecited by 8
  7. 2025Stable Thompson Sampling: Valid Inference via Variance Inflationcited by 8
  8. 2021Efficient Inference Without Trading-off Regret in Bandits: An Allocation Probability Test for Thompson Samplingcited by 7
  9. 2023Semi-parametric inference based on adaptively collected datacited by 7
  10. 2021Challenges in Statistical Analysis of Data Collected by a Bandit Algorithm: An Empirical Exploration in Applications to Adaptively Randomized Experimentscited by 6
  11. 2023Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resamplingcited by 6
  12. 2023Adaptive Linear Estimating Equationscited by 6
  13. 2022Integrating Reward Maximization and Population Estimation: Sequential Decision-Making for Internal Revenue Service Audit Selectioncited by 5
  14. 2022Reward Imputation with Sketching for Contextual Batched Banditscited by 5
  15. 2026Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directionscited by 5
  16. 2021Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learningcited by 4
  17. 2021Safe Data Collection for Offline and Online Policy Learningcited by 4
  18. 2024Best of Three Worlds: Adaptive Experimentation for Digital Marketing in Practicecited by 4
  19. 2024Logarithmic Neyman Regret for Adaptive Estimation of the Average Treatment Effectcited by 4
  20. 2024Allocation Stability and Wald Inference under Variance-Aware UCBcited by 4
  21. 2025Choosing the Better Bandit Algorithm under Data Sharing: When Do A/B Experiments Work?cited by 4
  22. 2025Adaptive Off-Policy Inference for M-Estimators Under Model Misspecificationcited by 4
  23. 2023Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inferencecited by 3
  24. 2024Batched Nonparametric Contextual Banditscited by 3
  25. 2024Off-policy estimation with adaptively collected data: the power of online learningcited by 3
  26. 2023Adaptive Experimentation at Scale: A Computational Framework for Flexible Batchescited by 2
  27. 2023Optimal tests following sequential experimentscited by 2
  28. 2024Oralytics Reinforcement Learning Algorithmcited by 2
  29. 2024MiWaves Reinforcement Learning Algorithmcited by 2
  30. 2025Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effectcited by 2
  31. 2025Statistical Inference in Reinforcement Learning: A Selective Surveycited by 2
  32. 2025Kernel Treatment Effects with Adaptively Collected Datacited by 2
  33. 2025Avoiding the Price of Adaptivity: Inference in Linear Contextual Bandits via Stabilitycited by 2
  34. 2026Optimism Stabilizes Thompson Sampling for Adaptive Inferencecited by 2
  35. 2026A Statistical Test for the Benefits of Personalizing Interventionscited by 2
  36. 2021Algorithms for Adaptive Experiments that Trade-off Statistical Analysis with Reward: Combining Uniform Random Assignment and Reward Maximizationcited by 1
  37. 2022Some performance considerations when using multi-armed bandit algorithms in the presence of missing datacited by 1
  38. 2022Entropy Regularization for Population Estimationcited by 1

50 most cited of 63 All 63 (JSON)