mlsanityData to 14 Oct 2026

A Two-armed Bandit Framework for A/B Testing

arXiv 2507.18118 ↗First posted 24 Jul 2025stat.ML cs.LG stat.APcited by 2

Not on the map. Why?

Abstract (arXiv)

A/B testing is widely used in modern technology companies for policy evaluation and product deployment, with the goal of comparing the outcomes under a newly-developed policy against a standard control. Various causal inference and reinforcement learning methods developed in the literature are applicable to A/B testing. This paper introduces a two-armed bandit framework designed to improve the power of existing approaches. The proposed procedure consists of three main steps: (i) employing doubly robust estimation to generate pseudo-outcomes, (ii) utilizing a two-armed bandit framework to construct the test statistic, and (iii) applying a permutation-based method to compute the p-value. We demonstrate the efficacy of the proposed method through asymptotic theories, numerical experiments and real-world data from a ridesharing company, showing its superior performance in comparison to existing methods.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 48 of its 98 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

48 of 98 references.

References · 48

  1. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
  2. 2019Batch Policy Learning under Constraintscited by 244
  3. 2018Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimationcited by 227
  4. 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
  5. 2016Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selectioncited by 121
  6. 2019Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processescited by 119
  7. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 95
  8. 2017Double/Debiased/Neyman Machine Learning of Treatment Effectscited by 76
  9. 2019Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learningcited by 75
  10. 2020Batch Policy Learning in Average Reward Markov Decision Processescited by 69
  11. 2016Balancing Covariates via Propensity Score Weightingcited by 65
  12. 2016Estimating Dynamic Treatment Regimes in Mobile Health Using V-learningcited by 65
36 more
  1. 2020Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settingscited by 65
  2. 2018Policy Evaluation and Optimization with Continuous Treatmentscited by 58
  3. 2016Approximate Residual Balancing: De-Biased Inference of Average Treatment Effects in High Dimensionscited by 50
  4. 2021A Deep Value-network Based Approach for Multi-Driver Order Dispatchingcited by 45
  5. 2013Graph cluster randomization: network exposure to multiple universescited by 42
  6. 2020Design and Analysis of Switchback Experimentscited by 38
  7. 2022Anytime-valid off-policy inference for contextual banditscited by 37
  8. 2018Counterfactual Mean Embeddingscited by 32
  9. 2020Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Frameworkcited by 29
  10. 2021Online Bootstrap Inference For Policy Evaluation in Reinforcement Learningcited by 26
  11. 2022Markovian Interference in Experimentscited by 22
  12. 2017Time series experiments and causal estimands: exact randomization tests and tradingcited by 21
  13. 2015Always Valid Inference: Bringing Sequential Analysis to A/B Testingcited by 20
  14. 2018Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structurescited by 20
  15. 2021Multiple Randomization Designs: Estimation and Inference with Interferencecited by 20
  16. 2023A/B Testing: A Systematic Literature Reviewcited by 16
  17. 2020Adaptive Experimental Design with Temporal Interference: A Maximum Likelihood Approachcited by 15
  18. 2021Rate-Optimal Cluster-Randomized Designs for Spatial Interferencecited by 14
  19. 2024Data-Driven Switchback Experiments: Theoretical Tradeoffs and Empirical Bayes Designscited by 12
  20. 2020Debiased Inverse Propensity Score Weighting for Estimation of Average Treatment Effects with High-Dimensional Confounderscited by 11
  21. 2021Graph-Based Equilibrium Metrics for Dynamic Supply-Demand Systems with Applications to Ride-sourcing Platformscited by 11
  22. 2023Optimal Treatment Allocation for Efficient Policy Evaluation in Sequential Decision Makingcited by 11
  23. 2021Average Direct and Indirect Causal Effects under Interferencecited by 10
  24. 2023An Efficient Doubly-Robust Test for the Kernel Treatment Effectcited by 10
  25. 2023Causal clustering: design of cluster experiments under network interferencecited by 10
  26. 2022Policy Evaluation for Temporal and/or Spatial Dependent Experimentscited by 8
  27. 2023Evaluating Dynamic Conditional Quantile Treatment Effects with Applications in Ridesharingcited by 7
  28. 2020Testing Mediation Effects Using Logic of Boolean Matricescited by 6
  29. 2024Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experimentscited by 6
  30. 2024Combining Experimental and Historical Data for Policy Evaluationcited by 6
  31. 2016Randomization and The Pernicious Effects of Limited Budgets on Auction Experimentscited by 5
  32. 2025Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approachcited by 5
  33. 2020Estimation and inference on high-dimensional individualized treatment rule in observational data using split-and-pooled de-correlated scorecited by 4
  34. 2020Toward Better Practice of Covariate Adjustment in Analyzing Randomized Clinical Trialscited by 2
  35. 2024Improving the Estimation of Lifetime Effects in A/B Testing via Treatment Localitycited by 2
  36. 2025Statistical Inference in Reinforcement Learning: A Selective Surveycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learningcited by 1
  2. 2026Robust Sequential Experimental Design for A/B Testingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing
  2. 2026Can AI Agents Simulate A/B Test Outcomes? A Validation Framework for Agentic Experimentation
  3. 2023Reinforcement Learning from Statistical Feedback: the Journey from AB Testing to ANT Testing
  4. 2025A Framework for Fair Evaluation of Variance-Aware Bandit Algorithms
  5. 2025A Reinforcement-Learning-Enhanced LLM Framework for Automated A/B Testing in Personalized Marketing
  6. 2025Causal Bandit Over Unknown Graphs: Upper Confidence Bounds With Backdoor Adjustment
  7. 2025Multi-Armed Bandits With Machine Learning-Generated Surrogate Rewards
  8. 2025Adaptive Reinforcement Learning for Dynamic Configuration Allocation in Pre-Production Testing