# A Deployed Online Reinforcement Learning Algorithm In An Oral Health Clinical Trial

arXiv: https://arxiv.org/abs/2409.02069 · HTML: https://mlsanity.com/paper/2409.02069/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 30 references that are in the map.
- First posted on arXiv 3 Sep 2024.

## Builds on (most cited first, 8 of 8)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity](https://mlsanity.com/paper/1909.03539/) (2019)
- [A Reinforcement Learning System to Encourage Physical Activity in Diabetes Patients](https://mlsanity.com/paper/1605.04070/) (2016)
- [Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care](https://mlsanity.com/paper/2208.07406/) (2022)
- [Replicable Bandits for Digital Health Interventions](https://mlsanity.com/paper/2407.15377/) (2024)
- [Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling](https://mlsanity.com/paper/2304.05365/) (2023)
- [Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health](https://mlsanity.com/paper/2310.18326/) (2023)
- [Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials](https://mlsanity.com/paper/2402.17003/) (2024)

## Built on by (most cited first, 11 of 11)

- [Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions](https://mlsanity.com/paper/2601.15353/) (2026)
- [Reinforcement Learning on Dyads to Enhance Medication Adherence](https://mlsanity.com/paper/2502.06835/) (2025)
- [SigmaScheduling: Uncertainty-Informed Scheduling of Decision Points for Intelligent Mobile Health Interventions](https://mlsanity.com/paper/2507.10798/) (2025)
- [Reproducible workflow for online AI in digital health](https://mlsanity.com/paper/2509.13499/) (2025)
- [A Diffusion-Model Subpopulation Digital Twin for Mobile Health Deployment: A Case Study on the HeartSteps Intervention](https://mlsanity.com/paper/2607.21403/) (2026)
- [Generalized Linear Markov Decision Process](https://mlsanity.com/paper/2506.00818/) (2025)
- [Can we use LLMs to bootstrap reinforcement learning? -- A case study in digital health behavior change](https://mlsanity.com/paper/2511.17630/) (2025)
- [Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding](https://mlsanity.com/paper/2602.00781/) (2026)
- [Deployment of AI-Assisted Interventions: Capacity Constraints and Noisy Compliance](https://mlsanity.com/paper/2604.14370/) (2026)
- [Optimizing Digital Therapeutic Interventions: Online Learning under Endogenous Adherence](https://mlsanity.com/paper/2605.24261/) (2026)
- [Bandit Simulation for Average Reward Inference](https://mlsanity.com/paper/2606.00913/) (2026)

## Abstract (arXiv)

Dental disease is a prevalent chronic condition associated with substantial financial burden, personal suffering, and increased risk of systemic diseases. Despite widespread recommendations for twice-daily tooth brushing, adherence to recommended oral self-care behaviors remains sub-optimal due to factors such as forgetfulness and disengagement. To address this, we developed Oralytics, a mHealth intervention system designed to complement clinician-delivered preventative care for marginalized individuals at risk for dental disease. Oralytics incorporates an online reinforcement learning algorithm to determine optimal times to deliver intervention prompts that encourage oral self-care behaviors. We have deployed Oralytics in a registered clinical trial. The deployment required careful design to manage challenges specific to the clinical trials setting in the U.S. In this paper, we (1) highlight key design decisions of the RL algorithm that address these challenges and (2) conduct a re-sampling analysis to evaluate algorithm design decisions. A second phase (randomized control trial) of Oralytics is planned to start in spring 2025.