# TSEC: a framework for online experimentation under experimental constraints

arXiv: https://arxiv.org/abs/2101.06592 · HTML: https://mlsanity.com/paper/2101.06592/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 56 references that are in the map.
- First posted on arXiv 17 Jan 2021.

## Builds on (most cited first, 10 of 10)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Minimal Exploration in Structured Stochastic Bandits](https://mlsanity.com/paper/1711.00400/) (2017)
- [Multi-Armed Bandits with Correlated Arms](https://mlsanity.com/paper/1911.03959/) (2019)
- [A Survey of Online Experiment Design with the Stochastic Multi-Armed Bandit](https://mlsanity.com/paper/1510.00757/) (2015)
- [Bounded Regret for Finite-Armed Structured Bandits](https://mlsanity.com/paper/1411.2919/) (2014)
- [Budget-Constrained Multi-Armed Bandits with Multiple Plays](https://mlsanity.com/paper/1711.05928/) (2017)
- [Regional Multi-Armed Bandits](https://mlsanity.com/paper/1802.07917/) (2018)
- [Practical Algorithms for Best-K Identification in Multi-Armed Bandits](https://mlsanity.com/paper/1705.06894/) (2017)
- [Adaptive Portfolio by Solving Multi-armed Bandit via Thompson Sampling](https://mlsanity.com/paper/1911.05309/) (2019)
- [Analysis-of-marginal-Tail-Means (ATM): a robust method for discrete black-box optimization](https://mlsanity.com/paper/1712.03589/) (2017)

## Abstract (arXiv)

Thompson sampling is a popular algorithm for solving multi-armed bandit problems, and has been applied in a wide range of applications, from website design to portfolio optimization. In such applications, however, the number of choices (or arms) N can be large, and the data needed to make adaptive decisions require expensive experimentation. One is then faced with the constraint of experimenting on only a small subset of K ll N arms within each time period, which poses a problem for traditional Thompson sampling. We propose a new Thompson Sampling under Experimental Constraints (TSEC) method, which addresses this so-called "arm budget constraint". TSEC makes use of a Bayesian interaction model with effect hierarchy priors, to model correlations between rewards on different arms. This fitted model is then integrated within Thompson sampling, to jointly identify a good subset of arms for experimentation and to allocate resources over these arms. We demonstrate the effectiveness of TSEC in two problems with arm budget constraints. The first is a simulated website optimization study, where TSEC shows noticeable improvements over industry benchmarks. The second is a portfolio optimization application on industry-based exchange-traded funds, where TSEC provides more consistent and greater wealth accumulation over standard investment strategies.