# Counterfactual inference in sequential experiments

arXiv: https://arxiv.org/abs/2202.06891 · HTML: https://mlsanity.com/paper/2202.06891/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 94 references that are in the map.
- First posted on arXiv 14 Feb 2022.

## Builds on (most cited first, 30 of 30)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Log-concave sampling: Metropolis-Hastings algorithms are fast](https://mlsanity.com/paper/1801.02309/) (2018)
- [Sequential Recommender Systems: Challenges, Progress and Prospects](https://mlsanity.com/paper/2001.04830/) (2019)
- [Confidence Intervals for Policy Evaluation in Adaptive Experiments](https://mlsanity.com/paper/1911.02768/) (2019)
- [Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity](https://mlsanity.com/paper/1909.03539/) (2019)
- [Inference for Batched Bandits](https://mlsanity.com/paper/2002.03217/) (2020)
- [Fast mixing of Metropolized Hamiltonian Monte Carlo: Benefits of multi-step gradients](https://mlsanity.com/paper/1905.12247/) (2019)
- [Kernel Thinning](https://mlsanity.com/paper/2105.05842/) (2021)
- [Causal Matrix Completion](https://mlsanity.com/paper/2109.15154/) (2021)
- [Synthetic Difference in Differences](https://mlsanity.com/paper/1812.09970/) (2018)
- [Statistical Inference with M-Estimators on Adaptively Collected Data](https://mlsanity.com/paper/2104.14074/) (2021)
- [Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Bandits](https://mlsanity.com/paper/2106.02029/) (2021)
- [Inference and Uncertainty Quantification for Noisy Matrix Completion](https://mlsanity.com/paper/1906.04159/) (2019)
- [A Reinforcement Learning System to Encourage Physical Activity in Diabetes Patients](https://mlsanity.com/paper/1605.04070/) (2016)
- [Singularity, Misspecification, and the Convergence Rate of EM](https://mlsanity.com/paper/1810.00828/) (2018)
- [Post-Contextual-Bandit Inference](https://mlsanity.com/paper/2106.00418/) (2021)
- [Generalized Kernel Thinning](https://mlsanity.com/paper/2110.01593/) (2021)
- [Stochastic Bandits for Multi-platform Budget Optimization in Online Advertising](https://mlsanity.com/paper/2103.10246/) (2021)
- [IntelligentPooling: Practical Thompson Sampling for mHealth](https://mlsanity.com/paper/2008.01571/) (2020)
- [Sharp Analysis of Expectation-Maximization for Weakly Identifiable Models](https://mlsanity.com/paper/1902.00194/) (2019)
- [Distribution Compression in Near-linear Time](https://mlsanity.com/paper/2111.07941/) (2021)
- [Stable discovery of interpretable subgroups via calibration in causal studies](https://mlsanity.com/paper/2008.10109/) (2020)
- [Fast MCMC sampling algorithms on polytopes](https://mlsanity.com/paper/1710.08165/) (2017)
- [Contextual Multi-Armed Bandits for Causal Marketing](https://mlsanity.com/paper/1810.01859/) (2018)
- [Instability, Computational Efficiency and Statistical Accuracy](https://mlsanity.com/paper/2005.11411/) (2020)
- [Curating a COVID-19 data repository and forecasting county-level death counts in the United States](https://mlsanity.com/paper/2005.07882/) (2020)
- [Panel Experiments and Dynamic Causal Effects: A Finite Population Perspective](https://mlsanity.com/paper/2003.09915/) (2020)
- [Revisiting minimum description length complexity in overparameterized models](https://mlsanity.com/paper/2006.10189/) (2020)
- [The power of online thinning in reducing discrepancy](https://mlsanity.com/paper/1608.02895/) (2016)
- [Uncertainty in Online Experiments with Dependent Data: An Evaluation of Bootstrap Methods](https://mlsanity.com/paper/1304.7406/) (2013)

## Built on by (most cited first, 13 of 13)

- [Off-policy Evaluation in Doubly Inhomogeneous Environments](https://mlsanity.com/paper/2306.08719/) (2023)
- [Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling](https://mlsanity.com/paper/2304.05365/) (2023)
- [Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning](https://mlsanity.com/paper/2412.05783/) (2024)
- [Doubly Robust Inference in Causal Latent Factor Models](https://mlsanity.com/paper/2402.11652/) (2024)
- [Distributional Matrix Completion via Nearest Neighbors in the Wasserstein Space](https://mlsanity.com/paper/2410.13112/) (2024)
- [Learning Counterfactual Distributions via Kernel Nearest Neighbors](https://mlsanity.com/paper/2410.13381/) (2024)
- [The Digital Transformation in Health: How AI Can Improve the Performance of Health Systems](https://mlsanity.com/paper/2409.16098/) (2024)
- [Two-Sided Nearest Neighbors: An adaptive and minimax optimal procedure for matrix completion](https://mlsanity.com/paper/2411.12965/) (2024)
- [Adaptively-weighted Nearest Neighbors for Matrix Completion](https://mlsanity.com/paper/2505.09612/) (2025)
- [Incentive-Aware Synthetic Control: Accurate Counterfactual Estimation via Incentivized Exploration](https://mlsanity.com/paper/2312.16307/) (2023)
- [N2: A Unified Python Package and Test Bench for Nearest Neighbor-Based Matrix Completion](https://mlsanity.com/paper/2506.04166/) (2025)
- [Counterfactual Forecasting for Panel Data](https://mlsanity.com/paper/2511.06189/) (2025)
- [C-kNN-LSH: A Nearest-Neighbor Algorithm for Sequential Counterfactual Inference](https://mlsanity.com/paper/2602.02371/) (2026)

## Abstract (arXiv)

We consider after-study statistical inference for sequentially designed experiments wherein multiple units are assigned treatments for multiple time points using treatment policies that adapt over time. Our goal is to provide inference guarantees for the counterfactual mean at the smallest possible scale -- mean outcome under different treatments for each unit and each time -- with minimal assumptions on the adaptive treatment policy. Without any structural assumptions on the counterfactual means, this challenging task is infeasible due to more unknowns than observed data points. To make progress, we introduce a latent factor model over the counterfactual means that serves as a non-parametric generalization of the non-linear mixed effects model and the bilinear latent factor model considered in prior works. For estimation, we use a non-parametric method, namely a variant of nearest neighbors, and establish a non-asymptotic high probability error bound for the counterfactual mean for each unit and each time. Under regularity conditions, this bound leads to asymptotically valid confidence intervals for the counterfactual mean as the number of units and time points grows to ∞ together at suitable rates. We illustrate our theory via several simulations and a case study involving data from a mobile health clinical trial HeartSteps.