# Statistical Inference After Adaptive Sampling for Longitudinal Data

arXiv: https://arxiv.org/abs/2202.07098 · HTML: https://mlsanity.com/paper/2202.07098/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2022.

## Built on by (most cited first, 12 of 12)

- [Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling](https://mlsanity.com/paper/2304.05365/) (2023)
- [reBandit: Random Effects based Online RL algorithm for Reducing Cannabis Use](https://mlsanity.com/paper/2402.17739/) (2024)
- [Optimal Adaptive Experimental Design for Estimating Treatment Effect](https://mlsanity.com/paper/2410.05552/) (2024)
- [Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions](https://mlsanity.com/paper/2601.15353/) (2026)
- [Adaptive Off-Policy Inference for M-Estimators Under Model Misspecification](https://mlsanity.com/paper/2509.14218/) (2025)
- [Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials](https://mlsanity.com/paper/2402.17003/) (2024)
- [Oralytics Reinforcement Learning Algorithm](https://mlsanity.com/paper/2406.13127/) (2024)
- [MiWaves Reinforcement Learning Algorithm](https://mlsanity.com/paper/2408.15076/) (2024)
- [Avoiding the Price of Adaptivity: Inference in Linear Contextual Bandits via Stability](https://mlsanity.com/paper/2512.20368/) (2025)
- [Effective Monitoring of Online Decision-Making Algorithms in Digital Intervention Implementation](https://mlsanity.com/paper/2409.10526/) (2024)
- [Bandit Simulation for Average Reward Inference](https://mlsanity.com/paper/2606.00913/) (2026)
- [Statistical Inference for Misspecified Contextual Bandits](https://mlsanity.com/paper/2606.22639/) (2026)

## Abstract (arXiv)

Online reinforcement learning and other adaptive sampling algorithms are increasingly used in digital intervention experiments to optimize treatment delivery for users over time. In this work, we focus on longitudinal user data collected by a large class of adaptive sampling algorithms that are designed to optimize treatment decisions online using accruing data from multiple users. Combining or "pooling" data across users allows adaptive sampling algorithms to potentially learn faster. However, by pooling, these algorithms induce dependence between the sampled user data trajectories; we show that this can cause standard variance estimators for i.i.d. data to underestimate the true variance of common estimators on this data type. We develop novel methods to perform a variety of statistical analyses on such adaptively sampled data via Z-estimation. Specifically, we introduce the textitadaptive sandwich variance estimator, a corrected sandwich estimator that leads to consistent variance estimates under adaptive sampling. Additionally, to prove our results we develop novel theoretical tools for empirical processes on non-i.i.d., adaptively sampled longitudinal data which may be of independent interest. This work is motivated by our efforts in designing experiments in which online reinforcement learning algorithms optimize treatment decisions, yet statistical inference is essential for conducting analyses after experiments conclude.