# Data-Driven Switchback Experiments: Theoretical Tradeoffs and Empirical Bayes Designs

arXiv: https://arxiv.org/abs/2406.06768 · HTML: https://mlsanity.com/paper/2406.06768/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 59 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Builds on (most cited first, 9 of 9)

- [Interference, Bias, and Variance in Two-Sided Marketplace Experimentation: Guidance for Platforms](https://mlsanity.com/paper/2104.12222/) (2021)
- [Switchback Experiments under Geometric Mixing](https://mlsanity.com/paper/2209.00197/) (2022)
- [Semiparametric Estimation of Treatment Effects in Observational Studies with Heterogeneous Partial Interference](https://mlsanity.com/paper/2107.12420/) (2021)
- [Experimenting under Stochastic Congestion](https://mlsanity.com/paper/2302.12093/) (2023)
- [Synthetic Controls for Experimental Design](https://mlsanity.com/paper/2108.02196/) (2021)
- [Modeling Interference Using Experiment Roll-out](https://mlsanity.com/paper/2305.10728/) (2023)
- [Central limit theorems via Stein's method for randomized experiments under interference](https://mlsanity.com/paper/1804.03105/) (2018)
- [Detecting Interference in A/B Testing with Increasing Allocation](https://mlsanity.com/paper/2211.03262/) (2022)
- [A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing](https://mlsanity.com/paper/2308.09790/) (2023)

## Built on by (most cited first, 12 of 12)

- [Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach](https://mlsanity.com/paper/2505.20130/) (2025)
- [Off-policy Evaluation with Deeply-abstracted States](https://mlsanity.com/paper/2406.19531/) (2024)
- [Choosing the Better Bandit Algorithm under Data Sharing: When Do A/B Experiments Work?](https://mlsanity.com/paper/2507.11891/) (2025)
- [A Two-armed Bandit Framework for A/B Testing](https://mlsanity.com/paper/2507.18118/) (2025)
- [Differences-in-Neighbors for Network Interference in Experiments](https://mlsanity.com/paper/2503.02271/) (2025)
- [Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learning](https://mlsanity.com/paper/2602.01853/) (2026)
- [Design Experiments to Compare Multi-armed Bandit Algorithms](https://mlsanity.com/paper/2603.05919/) (2026)
- [One-Sided Matrix Completion from Ultra-Sparse Samples](https://mlsanity.com/paper/2601.12213/) (2026)
- [Causal Effects with Unobserved Unit Types in Interacting Human-AI Systems](https://mlsanity.com/paper/2603.01339/) (2026)
- [Robust Sequential Experimental Design for A/B Testing](https://mlsanity.com/paper/2605.12899/) (2026)
- [Long-Context Demonstration Selection Using State Space Models](https://mlsanity.com/paper/2609.17888/) (2026)
- [Quantifying the Stability of Multi-Step Reasoning via Error Amplification](https://mlsanity.com/paper/2610.06404/) (2026)

## Abstract (arXiv)

We study the design and analysis of switchback experiments conducted on a single aggregate unit. The design problem is to partition the continuous time space into intervals and switch treatments between intervals, in order to minimize the estimation error of the treatment effect. We show that the estimation error depends on four factors: carryover effects, periodicity, serially correlated outcomes, and impacts from simultaneous experiments. We derive a rigorous bias-variance decomposition and show the tradeoffs of the estimation error from these factors. The decomposition provides three new insights in choosing a design: First, balancing the periodicity between treated and control intervals reduces the variance; second, switching less frequently reduces the bias from carryover effects while increasing the variance from correlated outcomes, and vice versa; third, randomizing interval start and end points reduces both bias and variance from simultaneous experiments. Combining these insights, we propose a new empirical Bayes design approach. This approach uses prior data and experiments for designing future experiments. We illustrate this approach using real data from a ride-sharing platform, yielding a design that reduces MSE by 33% compared to the status quo design used on the platform.