mlsanityData to 7 Oct 2026

Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments

arXiv 2403.17285 ↗First posted 26 Mar 2024stat.ML cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

A/B testing has become the gold standard for policy evaluation in modern technological industries. Motivated by the widespread use of switchback experiments in A/B testing, this paper conducts a comprehensive comparative analysis of various switchback designs in Markovian environments. Unlike many existing works which derive the optimal design based on specific and relatively simple estimators, our analysis covers a range of state-of-the-art estimators developed in the reinforcement learning (RL) literature. It reveals that the effectiveness of different switchback designs depends crucially on (i) the size of the carryover effect and (ii) the auto-correlations among reward errors over time. Meanwhile, these findings are estimator-agnostic, i.e., they apply to most RL estimators. Based on these insights, we provide a workflow to offer guidelines for practitioners on designing switchback experiments in A/B testing.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 73 of its 139 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

73 of 139 references.

References · 73

  1. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  2. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
  3. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  4. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 285
  5. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  6. 2018Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimationcited by 226
  7. 2019A Theoretical Analysis of Deep Q-Learningcited by 212
  8. 2019Minimax Weight and Q-Function Learning for Off-Policy Evaluationcited by 150
  9. 2019Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Samplingcited by 118
  10. 2019Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processescited by 118
  11. 2021Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Designcited by 103
  12. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 92
38 more
  1. 2019Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learningcited by 75
  2. 2020Batch Policy Learning in Average Reward Markov Decision Processescited by 69
  3. 2016Estimating Dynamic Treatment Regimes in Mobile Health Using V-learningcited by 65
  4. 2020Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settingscited by 65
  5. 2020CoinDICE: Off-Policy Confidence Interval Estimationcited by 64
  6. 2020Asymptotically Efficient Off-Policy Evaluation for Tabular Reinforcement Learningcited by 60
  7. 2021A Deep Value-network Based Approach for Multi-Driver Order Dispatchingcited by 45
  8. 2022Optimizing Sequential Experimental Design with Deep Reinforcement Learningcited by 45
  9. 2013Graph cluster randomization: network exposure to multiple universescited by 42
  10. 2019Off-Policy Estimation of Long-Term Average Outcomes with Applications to Mobile Healthcited by 42
  11. 2021Bootstrapping Fitted Q-Evaluation for Off-Policy Inferencecited by 40
  12. 2020Experimental Design in Two-Sided Platforms: An Analysis of Biascited by 39
  13. 2020Design and Analysis of Switchback Experimentscited by 37
  14. 2022Anytime-valid off-policy inference for contextual banditscited by 37
  15. 2022On Well-posedness and Minimax Optimal Rates of Nonparametric Q-function Estimation in Off-policy Evaluationcited by 33
  16. 2022Off-Policy Confidence Interval Estimation with Confounded Markov Decision Processcited by 31
  17. 2020Does the Markov Decision Process Fit the Data: Testing for the Markov Property in Sequential Decision Makingcited by 30
  18. 2020Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Frameworkcited by 29
  19. 2016Assessing Time-Varying Causal Effect Moderation in Mobile Healthcited by 27
  20. 2021Sample-Efficient Reinforcement Learning Is Feasible for Linearly Realizable MDPs with Limited Revisitingcited by 27
  21. 2017Data-Efficient Policy Evaluation Through Behavior Policy Searchcited by 25
  22. 2021Off-Policy Evaluation in Partially Observed Markov Decision Processes under Sequential Ignorabilitycited by 24
  23. 2021Interference, Bias, and Variance in Two-Sided Marketplace Experimentation: Guidance for Platformscited by 22
  24. 2022Markovian Interference in Experimentscited by 22
  25. 2017Time series experiments and causal estimands: exact randomization tests and tradingcited by 21
  26. 2021Multiple Randomization Designs: Estimation and Inference with Interferencecited by 20
  27. 2022An Instrumental Variable Approach to Confounded Off-Policy Evaluationcited by 20
  28. 2019Combining Parametric and Nonparametric Models for Off-Policy Evaluationcited by 17
  29. 2019More Efficient Off-Policy Evaluation through Regularized Targeted Learningcited by 17
  30. 2022Switchback Experiments under Geometric Mixingcited by 17
  31. 2017Auto-G-Computation of Causal Effects on a Networkcited by 16
  32. 2021Projected State-action Balancing Weights for Offline Reinforcement Learningcited by 16
  33. 2021Treatment Effects in Market Equilibriumcited by 16
  34. 2023A/B Testing: A Systematic Literature Reviewcited by 16
  35. 2020Adaptive Experimental Design with Temporal Interference: A Maximum Likelihood Approachcited by 15
  36. 2021Robust On-Policy Sampling for Data-Efficient Policy Evaluation in Reinforcement Learningcited by 15
  37. 2022Safe Exploration for Efficient Policy Evaluation and Comparisoncited by 15
  38. 2022Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Modelscited by 15

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approachcited by 5
  2. 2025Statistical Inference in Reinforcement Learning: A Selective Surveycited by 2
  3. 2025A Two-armed Bandit Framework for A/B Testingcited by 2
  4. 2025Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluationcited by 1
  5. 2026Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learningcited by 1
  6. 2026Robust Sequential Experimental Design for A/B Testingcited by 0