# Network experimentation at scale

arXiv: https://arxiv.org/abs/2012.08591 · HTML: https://mlsanity.com/paper/2012.08591/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2020.

## Built on by (most cited first, 8 of 8)

- [Learning Causal Effects on Hypergraphs](https://mlsanity.com/paper/2207.04049/) (2022)
- [Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach](https://mlsanity.com/paper/2505.20130/) (2025)
- [Graph Machine Learning based Doubly Robust Estimator for Network Causal Effects](https://mlsanity.com/paper/2403.11332/) (2024)
- [Uplift Modeling Under Limited Supervision](https://mlsanity.com/paper/2403.19289/) (2024)
- [Experimenting, Fast and Slow: Bayesian Optimization of Long-term Outcomes with Online Experiments](https://mlsanity.com/paper/2506.18744/) (2025)
- [Differences-in-Neighbors for Network Interference in Experiments](https://mlsanity.com/paper/2503.02271/) (2025)
- [A Simple Model to Estimate Sharing Effects in Social Networks](https://mlsanity.com/paper/2409.12203/) (2024)
- [Algorithm Adaptation Bias in Recommendation System Online Experiments](https://mlsanity.com/paper/2509.00199/) (2025)

## Abstract (arXiv)

We describe our framework, deployed at Facebook, that accounts for interference between experimental units through cluster-randomized experiments. We document this system, including the design and estimation procedures, and detail insights we have gained from the many experiments that have used this system at scale. We introduce a cluster-based regression adjustment that substantially improves precision for estimating global treatment effects as well as testing for interference as part of our estimation procedure. With this regression adjustment, we find that imbalanced clusters can better account for interference than balanced clusters without sacrificing accuracy. In addition, we show how logging exposure to a treatment can be used for additional variance reduction. Interference is a widely acknowledged issue with online field experiments, yet there is less evidence from real-world experiments demonstrating interference in online settings. We fill this gap by describing two case studies that capture significant network effects and highlight the value of this experimentation framework.