# A Comparison of Methods for Adaptive Experimentation

arXiv: https://arxiv.org/abs/2207.00683 · HTML: https://mlsanity.com/paper/2207.00683/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2022.

## Built on by (most cited first, 1 of 1)

- [Minimax Optimal Simple Regret in Two-Armed Best-Arm Identification](https://mlsanity.com/paper/2412.17753/) (2024)

## Abstract (arXiv)

We use a simulation study to compare three methods for adaptive experimentation: Thompson sampling, Tempered Thompson sampling, and Exploration sampling. We gauge the performance of each in terms of social welfare and estimation accuracy, and as a function of the number of experimental waves. We further construct a set of novel "hybrid" loss measures to identify which methods are optimal for researchers pursuing a combination of experimental aims. Our main results are: 1) the relative performance of Thompson sampling depends on the number of experimental waves, 2) Tempered Thompson sampling uniquely distributes losses across multiple experimental aims, and 3) in most cases, Exploration sampling performs similarly to random assignment.