# Challenges in Statistical Analysis of Data Collected by a Bandit Algorithm: An Empirical Exploration in Applications to Adaptively Randomized Experiments

arXiv: https://arxiv.org/abs/2103.12198 · HTML: https://mlsanity.com/paper/2103.12198/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 71 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Simple Bayesian Algorithms for Best Arm Identification](https://mlsanity.com/paper/1602.08448/) (2016)
- [Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges](https://mlsanity.com/paper/1507.08025/) (2015)
- [Confidence Intervals for Policy Evaluation in Adaptive Experiments](https://mlsanity.com/paper/1911.02768/) (2019)
- [Inference for Batched Bandits](https://mlsanity.com/paper/2002.03217/) (2020)
- [Accurate Inference for Adaptive Linear Models](https://mlsanity.com/paper/1712.06695/) (2017)
- [Why Adaptively Collected Data Have Negative Bias and How to Correct for It](https://mlsanity.com/paper/1708.01977/) (2017)
- [Power Constrained Bandits](https://mlsanity.com/paper/2004.06230/) (2020)
- [Thompson Sampling for Budgeted Multi-armed Bandits](https://mlsanity.com/paper/1505.00146/) (2015)
- [Are sample means in multi-armed bandits positively or negatively biased?](https://mlsanity.com/paper/1905.11397/) (2019)
- [Multi-Armed Bandits for Intelligent Tutoring Systems](https://mlsanity.com/paper/1310.3174/) (2013)
- [A framework for Multi-A(rmed)/B(andit) testing with online FDR control](https://mlsanity.com/paper/1706.05378/) (2017)
- [Personalized Course Sequence Recommendations](https://mlsanity.com/paper/1512.09176/) (2015)
- [On conditional versus marginal bias in multi-armed bandits](https://mlsanity.com/paper/2002.08422/) (2020)
- [Bayesian Best-Arm Identification for Selecting Influenza Mitigation Strategies](https://mlsanity.com/paper/1711.06299/) (2017)
- [A Bandit Approach to Multiple Testing with False Discovery Control](https://mlsanity.com/paper/1809.02235/) (2018)
- [Selecting a Biased-Coin Design](https://mlsanity.com/paper/1405.5051/) (2014)

## Built on by (most cited first, 6 of 6)

- [Multi-disciplinary fairness considerations in machine learning for clinical trials](https://mlsanity.com/paper/2205.08875/) (2022)
- [Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health](https://mlsanity.com/paper/2310.18326/) (2023)
- [Towards Regulatory-Confirmed Adaptive Clinical Trials: Machine Learning Opportunities and Solutions](https://mlsanity.com/paper/2503.09226/) (2025)
- [Infinite Action Contextual Bandits with Reusable Data Exhaust](https://mlsanity.com/paper/2302.08551/) (2023)
- [Opportunities for Adaptive Experiments to Enable Continuous Improvement in Computer Science Education](https://mlsanity.com/paper/2310.12324/) (2023)
- [Adaptive Experiments Under Data Sparse Settings: Applications for Educational Platforms](https://mlsanity.com/paper/2501.03999/) (2025)

## Abstract (arXiv)

Multi-armed bandit algorithms have been argued for decades as useful for adaptively randomized experiments. In such experiments, an algorithm varies which arms (e.g. alternative interventions to help students learn) are assigned to participants, with the goal of assigning higher-reward arms to as many participants as possible. We applied the bandit algorithm Thompson Sampling (TS) to run adaptive experiments in three university classes. Instructors saw great value in trying to rapidly use data to give their students in the experiments better arms (e.g. better explanations of a concept). Our deployment, however, illustrated a major barrier for scientists and practitioners to use such adaptive experiments: a lack of quantifiable insight into how much statistical analysis of specific real-world experiments is impacted (Pallmann et al, 2018; FDA, 2019), compared to traditional uniform random assignment. We therefore use our case study of the ubiquitous two-arm binary reward setting to empirically investigate the impact of using Thompson Sampling instead of uniform random assignment. In this setting, using common statistical hypothesis tests, we show that collecting data with TS can as much as double the False Positive Rate (FPR; incorrectly reporting differences when none exist) and the False Negative Rate (FNR; failing to report differences when they exist)...