# The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing

arXiv: https://arxiv.org/abs/2608.04432 · HTML: https://mlsanity.com/paper/2608.04432/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Builds on (most cited first, 10 of 10)

- [Graph cluster randomization: network exposure to multiple universes](https://mlsanity.com/paper/1305.6979/) (2013)
- [Experimental Design in Two-Sided Platforms: An Analysis of Bias](https://mlsanity.com/paper/2002.05670/) (2020)
- [Design and Analysis of Switchback Experiments](https://mlsanity.com/paper/2009.00148/) (2020)
- [Deconfounding Duration Bias in Watch-time Prediction for Video Recommendation](https://mlsanity.com/paper/2206.06003/) (2022)
- [Interference, Bias, and Variance in Two-Sided Marketplace Experimentation: Guidance for Platforms](https://mlsanity.com/paper/2104.12222/) (2021)
- [Multiple Randomization Designs: Estimation and Inference with Interference](https://mlsanity.com/paper/2112.13495/) (2021)
- [Long-Term Value of Exploration: Measurements, Findings and Algorithms](https://mlsanity.com/paper/2305.07764/) (2023)
- [A/B Testing for Recommender Systems in a Two-sided Marketplace](https://mlsanity.com/paper/2106.00762/) (2021)
- [When Does Interference Matter? Decision-Making in Platform Experiments](https://mlsanity.com/paper/2410.06580/) (2024)
- [Improving Ego-Cluster for Network Effect Measurement](https://mlsanity.com/paper/2308.05945/) (2023)

## Built on by (most cited first, 1 of 1)

- [Content Exploration Beyond the Feed: Creator Supply and the Shared Corpus](https://mlsanity.com/paper/2608.29430/) (2026)

## Abstract (arXiv)

On two-sided content platforms, symmetric two-sided isolation (assigning matched fractions of creators and viewers to isolated treatment and control submarkets) is widely used for creator-side and cold-start experiments because it removes cross-arm marketplace interference. Isolation, however, thins each viewer's candidate catalog, and intuition suggests the resulting engagement cost should fade as the platform grows: a small fraction of a vast catalog is still vast. We show that, in an order-statistics model of engagement, whether this intuition holds depends on the upper tail of match quality. Extreme-value theory yields tail-class loss laws with a sharp dichotomy: for light or bounded tails the loss vanishes as the candidate pool grows, whereas under heavy tails it converges to a size-independent constant, so expanding the candidate pool, even by orders of magnitude, does not asymptotically eliminate the cost. Evidence from two production experiments on a platform with millions of active creators is consistent with this picture: a pure A/A traffic sweep reveals a measurable, depth-graded engagement cost; a one-sided catalog ablation independently shows that per-viewer thinning contributes to the loss; and a tail index calibrated on the small exploration pool predicts an effect consistent with the one observed in the far larger full-catalog ablation. Isolation thus carries a price that experimenters should budget for, like any other cost. We give practitioners a preflight procedure that estimates it before launch, sizes traffic accordingly, and recommends a fallback design when the predicted cost exceeds a chosen tolerance.