# Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework

arXiv: https://arxiv.org/abs/2002.01711 · HTML: https://mlsanity.com/paper/2002.01711/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2020.

## Built on by (most cited first, 29 of 29)

- [A Review of Off-Policy Evaluation in Reinforcement Learning](https://mlsanity.com/paper/2212.06355/) (2022)
- [A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2111.06784/) (2021)
- [Online Bootstrap Inference For Policy Evaluation in Reinforcement Learning](https://mlsanity.com/paper/2108.03706/) (2021)
- [Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2205.13589/) (2022)
- [Markovian Interference in Experiments](https://mlsanity.com/paper/2206.02371/) (2022)
- [Causal Deep Learning](https://mlsanity.com/paper/2303.02186/) (2023)
- [Policy Evaluation for Temporal and/or Spatial Dependent Experiments](https://mlsanity.com/paper/2202.10887/) (2022)
- [Evaluating Dynamic Conditional Quantile Treatment Effects with Applications in Ridesharing](https://mlsanity.com/paper/2305.10187/) (2023)
- [Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments](https://mlsanity.com/paper/2403.17285/) (2024)
- [Combining Experimental and Historical Data for Policy Evaluation](https://mlsanity.com/paper/2406.00317/) (2024)
- [Quantile Off-Policy Evaluation via Deep Conditional Generative Learning](https://mlsanity.com/paper/2212.14466/) (2022)
- [Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach](https://mlsanity.com/paper/2505.20130/) (2025)
- [A Multi-Agent Reinforcement Learning Framework for Off-Policy Evaluation in Two-sided Markets](https://mlsanity.com/paper/2202.10574/) (2022)
- [Off-policy Evaluation with Deeply-abstracted States](https://mlsanity.com/paper/2406.19531/) (2024)
- [Clustered Switchback Designs for Experimentation Under Spatio-temporal Interference](https://mlsanity.com/paper/2312.15574/) (2023)
- [Statistical Inference in Reinforcement Learning: A Selective Survey](https://mlsanity.com/paper/2502.16195/) (2025)
- [Exploiting Similarities in A/B Testing with Off-Policy Estimation](https://mlsanity.com/paper/2506.10677/) (2025)
- [Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlations](https://mlsanity.com/paper/2507.09742/) (2025)
- [A Two-armed Bandit Framework for A/B Testing](https://mlsanity.com/paper/2507.18118/) (2025)
- [Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation](https://mlsanity.com/paper/2505.22492/) (2025)
- [Design Stability in Adaptive Experiments: Implications for Treatment Effect Estimation](https://mlsanity.com/paper/2510.22351/) (2025)
- [Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learning](https://mlsanity.com/paper/2602.01853/) (2026)
- [Evaluating Uplift Modeling under Structural Biases: Insights into Metric Stability and Model Robustness](https://mlsanity.com/paper/2603.20775/) (2026)
- [Always Valid Risk Monitoring for Online Matrix Completion](https://mlsanity.com/paper/2211.10363/) (2022)
- [Deep Spectral Q-learning with Application to Mobile Health](https://mlsanity.com/paper/2301.00927/) (2023)
- [Short-Long Policy Evaluation with Novel Actions](https://mlsanity.com/paper/2407.03674/) (2024)
- [Predicting Long Term Sequential Policy Value Using Softer Surrogates](https://mlsanity.com/paper/2412.20638/) (2024)
- [Generalized Fitted Q-Iteration with Clustered Data](https://mlsanity.com/paper/2510.03912/) (2025)
- [Robust Sequential Experimental Design for A/B Testing](https://mlsanity.com/paper/2605.12899/) (2026)

## Abstract (arXiv)

A/B testing, or online experiment is a standard business strategy to compare a new product with an old one in pharmaceutical, technological, and traditional industries. Major challenges arise in online experiments of two-sided marketplace platforms (e.g., Uber) where there is only one unit that receives a sequence of treatments over time. In those experiments, the treatment at a given time impacts current outcome as well as future outcomes. The aim of this paper is to introduce a reinforcement learning framework for carrying A/B testing in these experiments, while characterizing the long-term treatment effects. Our proposed testing procedure allows for sequential monitoring and online updating. It is generally applicable to a variety of treatment designs in different industries. In addition, we systematically investigate the theoretical properties (e.g., size and power) of our testing procedure. Finally, we apply our framework to both simulated data and a real-world data example obtained from a technological company to illustrate its advantage over the current practice. A Python implementation of our test is available at https://github.com/callmespring/CausalRL.