# Design and Analysis of Switchback Experiments

arXiv: https://arxiv.org/abs/2009.00148 · HTML: https://mlsanity.com/paper/2009.00148/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2020.

## Built on by (most cited first, 30 of 38)

- [Markovian Interference in Experiments](https://mlsanity.com/paper/2206.02371/) (2022)
- [Correcting for Interference in Experiments: A Case Study at Douyin](https://mlsanity.com/paper/2305.02542/) (2023)
- [Multi-Armed Bandits with Interference](https://mlsanity.com/paper/2402.01845/) (2024)
- [Synthetically Controlled Bandits](https://mlsanity.com/paper/2202.07079/) (2022)
- [Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments](https://mlsanity.com/paper/2403.17285/) (2024)
- [Combining Experimental and Historical Data for Policy Evaluation](https://mlsanity.com/paper/2406.00317/) (2024)
- [Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach](https://mlsanity.com/paper/2505.20130/) (2025)
- [Off-policy Evaluation with Deeply-abstracted States](https://mlsanity.com/paper/2406.19531/) (2024)
- [Double Machine Learning for Causal Inference under Shared-State Interference](https://mlsanity.com/paper/2504.08836/) (2025)
- [Choosing the Better Bandit Algorithm under Data Sharing: When Do A/B Experiments Work?](https://mlsanity.com/paper/2507.11891/) (2025)
- [A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing](https://mlsanity.com/paper/2308.09790/) (2023)
- [Tackling Interference Induced by Data Training Loops in A/B Tests: A Weighted Training Approach](https://mlsanity.com/paper/2310.17496/) (2023)
- [Clustered Switchback Designs for Experimentation Under Spatio-temporal Interference](https://mlsanity.com/paper/2312.15574/) (2023)
- [Simulating and Experimenting with Social Media Mobilization Using LLM Agents](https://mlsanity.com/paper/2510.26494/) (2025)
- [Causal Deepsets for Off-policy Evaluation under Spatial or Spatio-temporal Interferences](https://mlsanity.com/paper/2407.17910/) (2024)
- [A Two-armed Bandit Framework for A/B Testing](https://mlsanity.com/paper/2507.18118/) (2025)
- [Decision Support for Marketplace Policies under Incomplete Evidence: From Replay to Launch Readiness](https://mlsanity.com/paper/2605.12840/) (2026)
- [A Unified Representation Framework for Rideshare Marketplace Equilibrium and Efficiency](https://mlsanity.com/paper/2302.14358/) (2023)
- [Privacy Preserving Adaptive Experiment Design](https://mlsanity.com/paper/2401.08224/) (2024)
- [Differences-in-Neighbors for Network Interference in Experiments](https://mlsanity.com/paper/2503.02271/) (2025)
- [On Evolution-Based Models for Experimentation Under Interference](https://mlsanity.com/paper/2511.21675/) (2025)
- [Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learning](https://mlsanity.com/paper/2602.01853/) (2026)
- [Design Experiments to Compare Multi-armed Bandit Algorithms](https://mlsanity.com/paper/2603.05919/) (2026)
- [Support-aware offline policy selection for advertising marketplaces](https://mlsanity.com/paper/2605.21736/) (2026)
- [Choosing Online Experiment Designs under Interference in Ads, Recommendations, and Member-Experience Systems](https://mlsanity.com/paper/2605.25290/) (2026)
- [The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing](https://mlsanity.com/paper/2608.04432/) (2026)
- [Near-Optimal Experimental Design Under the Budget Constraint in Online Platforms](https://mlsanity.com/paper/2302.05005/) (2023)
- [Validation of massively-parallel adaptive testing using dynamic control matching](https://mlsanity.com/paper/2305.01334/) (2023)
- [Causal Estimation of User Learning in Personalized Systems](https://mlsanity.com/paper/2306.00485/) (2023)
- [Beyond ATE: Multi-Criteria Design for A/B Testing](https://mlsanity.com/paper/2509.05864/) (2025)

## Abstract (arXiv)

Switchback experiments, where a firm sequentially exposes an experimental unit to random treatments, are among the most prevalent designs used in the technology sector, with applications ranging from ride-hailing platforms to online marketplaces. Although practitioners have widely adopted this technique, the derivation of the optimal design has been elusive, hindering practitioners from drawing valid causal conclusions with enough statistical power. We address this limitation by deriving the optimal design of switchback experiments under a range of different assumptions on the order of the carryover effect -- the length of time a treatment persists in impacting the outcome. We cast the optimal experimental design problem as a minimax discrete optimization problem, identify the worst-case adversarial strategy, establish structural results, and solve the reduced problem via a continuous relaxation. For switchback experiments conducted under the optimal design, we provide two approaches for performing inference. The first provides exact randomization based p-values, and the second uses a new finite population central limit theorem to conduct conservative hypothesis tests and build confidence intervals. We further provide theoretical results when the order of the carryover effect is misspecified and provide a data-driven procedure to identify the order of the carryover effect. We conduct extensive simulations to study the numerical performance and empirical properties of our results, and conclude with practical suggestions.