mlsanityData to 7 Oct 2026

Robust Sequential Experimental Design for A/B Testing

arXiv 2605.12899 ↗First posted 13 May 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Experimental design has emerged as a powerful approach for improving the sample efficiency of A/B testing, yet existing designs rely critically on correctly specified models. We study robust sequential experimental design under model misspecification and develop a unified framework that covers both contextual bandit and dynamic settings. Theoretically, we prove that our design bounds the worst-case mean squared error of the estimated treatment effect. Empirically, we demonstrate the effectiveness of the proposed approach using synthetic and real-world datasets from a leading technology company.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 80 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

32 of 80 references.

References · 32

  1. 2021A Deep Value-network Based Approach for Multi-Driver Order Dispatchingcited by 45
  2. 2020Design and Analysis of Switchback Experimentscited by 37
  3. 2022Anytime-valid off-policy inference for contextual banditscited by 37
  4. 2022Off-Policy Confidence Interval Estimation with Confounded Markov Decision Processcited by 31
  5. 2020Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Frameworkcited by 29
  6. 2016Sequential Bayesian optimal experimental design via approximate dynamic programmingcited by 27
  7. 2022Markovian Interference in Experimentscited by 22
  8. 2021Multiple Randomization Designs: Estimation and Inference with Interferencecited by 20
  9. 2022An Instrumental Variable Approach to Confounded Off-Policy Evaluationcited by 20
  10. 2022Switchback Experiments under Geometric Mixingcited by 17
  11. 2023A/B Testing: A Systematic Literature Reviewcited by 16
  12. 2020Adaptive Experimental Design with Temporal Interference: A Maximum Likelihood Approachcited by 15
20 more
  1. 2022Conformal Off-policy Predictioncited by 14
  2. 2023Correcting for Interference in Experiments: A Case Study at Douyincited by 12
  3. 2024Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choicescited by 12
  4. 2023Optimal Treatment Allocation for Efficient Policy Evaluation in Sequential Decision Makingcited by 11
  5. 2024Data-Driven Switchback Experiments: Theoretical Tradeoffs and Empirical Bayes Designscited by 11
  6. 2022Policy Evaluation for Temporal and/or Spatial Dependent Experimentscited by 8
  7. 2023Evaluating Dynamic Conditional Quantile Treatment Effects with Applications in Ridesharingcited by 7
  8. 2020Cluster-Adaptive Network A/B Testing: From Randomization to Estimationcited by 6
  9. 2024Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experimentscited by 6
  10. 2024Combining Experimental and Historical Data for Policy Evaluationcited by 6
  11. 2019Synthetic learner: model-free inference on treatments over timecited by 5
  12. 2025Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approachcited by 5
  13. 2022Detecting Interference in A/B Testing with Increasing Allocationcited by 4
  14. 2023Leveraging covariate adjustments at scale in online A/B testingcited by 4
  15. 2021An Online Sequential Test for Qualitative Treatment Effectscited by 2
  16. 2025A Two-armed Bandit Framework for A/B Testingcited by 2
  17. 2026Budgeted Active Experimentation for Treatment Effect Estimation from Observational and Randomized Datacited by 2
  18. 2014Model-Robust Designs for Quantile Regressioncited by 1
  19. 2025Estimation of Treatment Effects Under Nonstationarity via the Truncated Policy Gradient Estimatorcited by 1
  20. 2026Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learningcited by 1

Built on by

Not cited yet.