# Optimal Treatment Allocation for Efficient Policy Evaluation in Sequential Decision Making

arXiv: https://arxiv.org/abs/2311.02532 · HTML: https://mlsanity.com/paper/2311.02532/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2023.

## Built on by (most cited first, 11 of 11)

- [Doubly Robust Alignment for Large Language Models](https://mlsanity.com/paper/2506.01183/) (2025)
- [Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments](https://mlsanity.com/paper/2403.17285/) (2024)
- [Logarithmic Neyman Regret for Adaptive Estimation of the Average Treatment Effect](https://mlsanity.com/paper/2411.14341/) (2024)
- [Moments of Causal Effects](https://mlsanity.com/paper/2505.04971/) (2025)
- [Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect](https://mlsanity.com/paper/2502.04673/) (2025)
- [Statistical Inference in Reinforcement Learning: A Selective Survey](https://mlsanity.com/paper/2502.16195/) (2025)
- [A Two-armed Bandit Framework for A/B Testing](https://mlsanity.com/paper/2507.18118/) (2025)
- [Multi-Metric Adaptive Experimental Design Under a Fixed Budget with Validation](https://mlsanity.com/paper/2506.03062/) (2025)
- [Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learning](https://mlsanity.com/paper/2602.01853/) (2026)
- [Bounds and Identification of Joint Probabilities of Potential Outcomes and Observed Variables under Monotonicity Assumptions](https://mlsanity.com/paper/2602.18762/) (2026)
- [Robust Sequential Experimental Design for A/B Testing](https://mlsanity.com/paper/2605.12899/) (2026)

## Abstract (arXiv)

A/B testing is critical for modern technological companies to evaluate the effectiveness of newly developed products against standard baselines. This paper studies optimal designs that aim to maximize the amount of information obtained from online experiments to estimate treatment effects accurately. We propose three optimal allocation strategies in a dynamic setting where treatments are sequentially assigned over time. These strategies are designed to minimize the variance of the treatment effect estimator when data follow a non-Markov decision process or a (time-varying) Markov decision process. We further develop estimation procedures based on existing off-policy evaluation (OPE) methods and conduct extensive experiments in various environments to demonstrate the effectiveness of the proposed methodologies. In theory, we prove the optimality of the proposed treatment allocation design and establish upper bounds for the mean squared errors of the resulting treatment effect estimators.