# Replicable Bandits for Digital Health Interventions

arXiv: https://arxiv.org/abs/2407.15377 · HTML: https://mlsanity.com/paper/2407.15377/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2024.

## Built on by (most cited first, 8 of 8)

- [A Deployed Online Reinforcement Learning Algorithm In An Oral Health Clinical Trial](https://mlsanity.com/paper/2409.02069/) (2024)
- [Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions](https://mlsanity.com/paper/2601.15353/) (2026)
- [Reproducible workflow for online AI in digital health](https://mlsanity.com/paper/2509.13499/) (2025)
- [Learning with Incomplete Context: Linear Contextual Bandits with Pretrained Imputation](https://mlsanity.com/paper/2510.09908/) (2025)
- [Deployment of AI-Assisted Interventions: Capacity Constraints and Noisy Compliance](https://mlsanity.com/paper/2604.14370/) (2026)
- [Replicable Bandits with UCB based Exploration](https://mlsanity.com/paper/2604.20024/) (2026)
- [Bandit Simulation for Average Reward Inference](https://mlsanity.com/paper/2606.00913/) (2026)
- [Statistical Inference for Misspecified Contextual Bandits](https://mlsanity.com/paper/2606.22639/) (2026)

## Abstract (arXiv)

Adaptive treatment assignment algorithms, such as bandit algorithms, are increasingly used in digital health intervention clinical trials. Frequently, the data collected from these trials is used to conduct causal inference and related data analyses to decide how to refine the intervention, and whether to roll-out the intervention more broadly. This work studies inference for estimands that depend on the adaptive algorithm itself; a simple example is the mean reward under the adaptive algorithm. Specifically, we investigate the replicability of statistical analyses concerning such estimands when using data from trials deploying adaptive treatment assignment algorithms. We demonstrate that many standard statistical estimators can be inconsistent and fail to be replicable across repetitions of the clinical trial, even as the sample size grows large. We show that this non-replicability is intimately related to properties of the adaptive algorithm itself. We introduce a formal definition of a "replicable bandit algorithm" and prove that under such algorithms, a wide variety of common statistical estimators are guaranteed to be consistent and asymptotically normal. We present both theoretical results and simulation studies based on a mobile health oral health self-care intervention. Our findings underscore the importance of designing adaptive algorithms with replicability in mind, especially for settings like digital health, where deployment decisions rely heavily on replicated evidence. We conclude by discussing open questions on the connections between algorithm design, statistical inference, and experimental replicability.