# Randomization and The Pernicious Effects of Limited Budgets on Auction Experiments

arXiv: https://arxiv.org/abs/1605.09171 · HTML: https://mlsanity.com/paper/1605.09171/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2016.

## Built on by (most cited first, 5 of 5)

- [Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments](https://mlsanity.com/paper/2403.17285/) (2024)
- [Double Machine Learning for Causal Inference under Shared-State Interference](https://mlsanity.com/paper/2504.08836/) (2025)
- [Tackling Interference Induced by Data Training Loops in A/B Tests: A Weighted Training Approach](https://mlsanity.com/paper/2310.17496/) (2023)
- [A Two-armed Bandit Framework for A/B Testing](https://mlsanity.com/paper/2507.18118/) (2025)
- [Near-Optimal Experimental Design Under the Budget Constraint in Online Platforms](https://mlsanity.com/paper/2302.05005/) (2023)

## Abstract (arXiv)

Buyers (e.g., advertisers) often have limited financial and processing resources, and so their participation in auctions is throttled. Changes to auctions may affect bids or throttling and any change may affect what winners pay. This paper shows that if an A/B experiment affects only bids, then the observed treatment effect is unbiased when all the bidders in an auction are randomly assigned to A or B but it can be severely biased otherwise, even in the absence of throttling. Experiments that affect throttling algorithms can also be badly biased, but the bias can be substantially reduced if the budget for each advertiser in the experiment is allocated to separate pots for the A and B arms of the experiment.