# Near-Optimal Experimental Design Under the Budget Constraint in Online Platforms

arXiv: https://arxiv.org/abs/2302.05005 · HTML: https://mlsanity.com/paper/2302.05005/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 10 Feb 2023.

## Builds on (most cited first, 17 of 17)

- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Graph cluster randomization: network exposure to multiple universes](https://mlsanity.com/paper/1305.6979/) (2013)
- [Design and analysis of experiments in networks: Reducing bias from interference](https://mlsanity.com/paper/1404.7530/) (2014)
- [Experimental Design in Two-Sided Platforms: An Analysis of Bias](https://mlsanity.com/paper/2002.05670/) (2020)
- [Design and Analysis of Switchback Experiments](https://mlsanity.com/paper/2009.00148/) (2020)
- [Fast Algorithms for Online Stochastic Convex Programming](https://mlsanity.com/paper/1410.7596/) (2014)
- [Primal Beats Dual on Online Packing LPs in the Random-Order Model](https://mlsanity.com/paper/1311.2578/) (2013)
- [Causal Network Motifs: Identifying Heterogeneous Spillover Effects in A/B Tests](https://mlsanity.com/paper/2010.09911/) (2020)
- [Reducing Interference Bias in Online Marketplace Pricing Experiments](https://mlsanity.com/paper/2004.12489/) (2020)
- [How the Experts Algorithm Can Help Solve LPs Online](https://mlsanity.com/paper/1407.5298/) (2014)
- [Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment](https://mlsanity.com/paper/1305.6156/) (2013)
- [Testing for arbitrary interference on experimentation platforms](https://mlsanity.com/paper/1704.01190/) (2017)
- [A Graph-Theoretic Approach to Randomization Tests of Causal Effects Under General Interference](https://mlsanity.com/paper/1910.10862/) (2019)
- [Adaptive normalization for IPW estimation](https://mlsanity.com/paper/2106.07695/) (2021)
- [A Field Guide to Personalized Reserve Prices](https://mlsanity.com/paper/1602.07720/) (2016)
- [Randomization and The Pernicious Effects of Limited Budgets on Auction Experiments](https://mlsanity.com/paper/1605.09171/) (2016)
- [Adwords in a Panorama](https://mlsanity.com/paper/2009.04114/) (2020)

## Abstract (arXiv)

A/B testing, or controlled experiments, is the gold standard approach to causally compare the performance of algorithms on online platforms. However, conventional Bernoulli randomization in A/B testing faces many challenges such as spillover and carryover effects. Our study focuses on another challenge, especially for A/B testing on two-sided platforms -- budget constraints. Buyers on two-sided platforms often have limited budgets, where the conventional A/B testing may be infeasible to be applied, partly because two variants of allocation algorithms may conflict and lead some buyers to exceed their budgets if they are implemented simultaneously. We develop a model to describe two-sided platforms where buyers have limited budgets. We then provide an optimal experimental design that guarantees small bias and minimum variance. Bias is lower when there is more budget and a higher supply-demand rate. We test our experimental design on both synthetic data and real-world data, which verifies the theoretical results and shows our advantage compared to Bernoulli randomization.