# IntelligentPooling: Practical Thompson Sampling for mHealth

arXiv: https://arxiv.org/abs/2008.01571 · HTML: https://mlsanity.com/paper/2008.01571/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2020.

## Built on by (most cited first, 24 of 24)

- [Metadata-based Multi-Task Bandits with Bayesian Hierarchical Models](https://mlsanity.com/paper/2108.06422/) (2021)
- [Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines](https://mlsanity.com/paper/2206.03944/) (2022)
- [Mixed-Effect Thompson Sampling](https://mlsanity.com/paper/2205.15124/) (2022)
- [Counterfactual inference in sequential experiments](https://mlsanity.com/paper/2202.06891/) (2022)
- [Bandit Algorithms for Precision Medicine](https://mlsanity.com/paper/2108.04782/) (2021)
- [Reinforcement Learning in Practice: Opportunities and Challenges](https://mlsanity.com/paper/2202.11296/) (2022)
- [Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions](https://mlsanity.com/paper/2203.02605/) (2022)
- [Data-pooling Reinforcement Learning for Personalized Healthcare Intervention](https://mlsanity.com/paper/2211.08998/) (2022)
- [RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions](https://mlsanity.com/paper/2312.06403/) (2023)
- [Revisiting Weighted Strategy for Non-stationary Parametric Bandits and MDPs](https://mlsanity.com/paper/2601.01069/) (2026)
- [Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions](https://mlsanity.com/paper/2601.15353/) (2026)
- [Thompson sampling for zero-inflated count outcomes with an application to the Drink Less mobile health study](https://mlsanity.com/paper/2311.14359/) (2023)
- [Oralytics Reinforcement Learning Algorithm](https://mlsanity.com/paper/2406.13127/) (2024)
- [Generator-Mediated Bandits: Thompson Sampling for GenAI-Powered Adaptive Interventions](https://mlsanity.com/paper/2505.16311/) (2025)
- [A Deep Bayesian Bandits Approach for Anticancer Therapy: Exploration via Functional Prior](https://mlsanity.com/paper/2205.02944/) (2022)
- [Revisiting Weighted Strategy for Non-stationary Parametric Bandits](https://mlsanity.com/paper/2303.02691/) (2023)
- [Designing and evaluating an online reinforcement learning agent for physical exercise recommendations in N-of-1 trials](https://mlsanity.com/paper/2309.14156/) (2023)
- [Adaptive Interventions with User-Defined Goals for Health Behavior Change](https://mlsanity.com/paper/2311.09483/) (2023)
- [The Nah Bandit: Modeling User Non-compliance in Recommendation Systems](https://mlsanity.com/paper/2408.07897/) (2024)
- [Can we use LLMs to bootstrap reinforcement learning? -- A case study in digital health behavior change](https://mlsanity.com/paper/2511.17630/) (2025)
- [Optimizing Digital Therapeutic Interventions: Online Learning under Endogenous Adherence](https://mlsanity.com/paper/2605.24261/) (2026)
- [Nonstationary Generalized Linear Bandits with Discounted Online Mirror Descent](https://mlsanity.com/paper/2605.25590/) (2026)
- [Personalizing Large Language Model Agents with Small Policy Models](https://mlsanity.com/paper/2608.00215/) (2026)
- [Dominant Arm Identification with Mixing and Recycling Observed Samples](https://mlsanity.com/paper/2608.01545/) (2026)

## Abstract (arXiv)

In mobile health (mHealth) smart devices deliver behavioral treatments repeatedly over time to a user with the goal of helping the user adopt and maintain healthy behaviors. Reinforcement learning appears ideal for learning how to optimally make these sequential treatment decisions. However, significant challenges must be overcome before reinforcement learning can be effectively deployed in a mobile healthcare setting. In this work we are concerned with the following challenges: 1) individuals who are in the same context can exhibit differential response to treatments 2) only a limited amount of data is available for learning on any one individual, and 3) non-stationary responses to treatment. To address these challenges we generalize Thompson-Sampling bandit algorithms to develop IntelligentPooling. IntelligentPooling learns personalized treatment policies thus addressing challenge one. To address the second challenge, IntelligentPooling updates each user's degree of personalization while making use of available data on other users to speed up learning. Lastly, IntelligentPooling allows responsivity to vary as a function of a user's time since beginning treatment, thus addressing challenge three. We show that IntelligentPooling achieves an average of 26% lower regret than state-of-the-art. We demonstrate the promise of this approach and its ability to learn from even a small group of users in a live clinical trial.