# Robust Sequential Experimental Design for A/B Testing

arXiv: https://arxiv.org/abs/2605.12899 · HTML: https://mlsanity.com/paper/2605.12899/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 80 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 30 of 32)

- [A Deep Value-network Based Approach for Multi-Driver Order Dispatching](https://mlsanity.com/paper/2106.04493/) (2021)
- [Design and Analysis of Switchback Experiments](https://mlsanity.com/paper/2009.00148/) (2020)
- [Anytime-valid off-policy inference for contextual bandits](https://mlsanity.com/paper/2210.10768/) (2022)
- [Off-Policy Confidence Interval Estimation with Confounded Markov Decision Process](https://mlsanity.com/paper/2202.10589/) (2022)
- [Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework](https://mlsanity.com/paper/2002.01711/) (2020)
- [Sequential Bayesian optimal experimental design via approximate dynamic programming](https://mlsanity.com/paper/1604.08320/) (2016)
- [Markovian Interference in Experiments](https://mlsanity.com/paper/2206.02371/) (2022)
- [Multiple Randomization Designs: Estimation and Inference with Interference](https://mlsanity.com/paper/2112.13495/) (2021)
- [An Instrumental Variable Approach to Confounded Off-Policy Evaluation](https://mlsanity.com/paper/2212.14468/) (2022)
- [Switchback Experiments under Geometric Mixing](https://mlsanity.com/paper/2209.00197/) (2022)
- [A/B Testing: A Systematic Literature Review](https://mlsanity.com/paper/2308.04929/) (2023)
- [Adaptive Experimental Design with Temporal Interference: A Maximum Likelihood Approach](https://mlsanity.com/paper/2006.05591/) (2020)
- [Conformal Off-policy Prediction](https://mlsanity.com/paper/2206.06711/) (2022)
- [Correcting for Interference in Experiments: A Case Study at Douyin](https://mlsanity.com/paper/2305.02542/) (2023)
- [Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choices](https://mlsanity.com/paper/2403.03589/) (2024)
- [Data-Driven Switchback Experiments: Theoretical Tradeoffs and Empirical Bayes Designs](https://mlsanity.com/paper/2406.06768/) (2024)
- [Optimal Treatment Allocation for Efficient Policy Evaluation in Sequential Decision Making](https://mlsanity.com/paper/2311.02532/) (2023)
- [Policy Evaluation for Temporal and/or Spatial Dependent Experiments](https://mlsanity.com/paper/2202.10887/) (2022)
- [Evaluating Dynamic Conditional Quantile Treatment Effects with Applications in Ridesharing](https://mlsanity.com/paper/2305.10187/) (2023)
- [Cluster-Adaptive Network A/B Testing: From Randomization to Estimation](https://mlsanity.com/paper/2008.08648/) (2020)
- [Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments](https://mlsanity.com/paper/2403.17285/) (2024)
- [Combining Experimental and Historical Data for Policy Evaluation](https://mlsanity.com/paper/2406.00317/) (2024)
- [Synthetic learner: model-free inference on treatments over time](https://mlsanity.com/paper/1904.01490/) (2019)
- [Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach](https://mlsanity.com/paper/2505.20130/) (2025)
- [Detecting Interference in A/B Testing with Increasing Allocation](https://mlsanity.com/paper/2211.03262/) (2022)
- [Leveraging covariate adjustments at scale in online A/B testing](https://mlsanity.com/paper/2305.01109/) (2023)
- [An Online Sequential Test for Qualitative Treatment Effects](https://mlsanity.com/paper/2111.03908/) (2021)
- [A Two-armed Bandit Framework for A/B Testing](https://mlsanity.com/paper/2507.18118/) (2025)
- [Budgeted Active Experimentation for Treatment Effect Estimation from Observational and Randomized Data](https://mlsanity.com/paper/2602.22021/) (2026)
- [Model-Robust Designs for Quantile Regression](https://mlsanity.com/paper/1403.1638/) (2014)

## Abstract (arXiv)

Experimental design has emerged as a powerful approach for improving the sample efficiency of A/B testing, yet existing designs rely critically on correctly specified models. We study robust sequential experimental design under model misspecification and develop a unified framework that covers both contextual bandit and dynamic settings. Theoretically, we prove that our design bounds the worst-case mean squared error of the estimated treatment effect. Empirically, we demonstrate the effectiveness of the proposed approach using synthetic and real-world datasets from a leading technology company.