# Offline Contextual Multi-armed Bandits for Mobile Health Interventions: A Case Study on Emotion Regulation

arXiv: https://arxiv.org/abs/2008.09472 · HTML: https://mlsanity.com/paper/2008.09472/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2020.

## Built on by (most cited first, 13 of 13)

- [Bandit Algorithms for Precision Medicine](https://mlsanity.com/paper/2108.04782/) (2021)
- [Pre-Trained AI Model Assisted Online Decision-Making under Missing Covariates: A Theoretical Perspective](https://mlsanity.com/paper/2507.07852/) (2025)
- [Risk-Averse Multi-Armed Bandits with Unobserved Confounders: A Case Study in Emotion Regulation in Mobile Health](https://mlsanity.com/paper/2209.04356/) (2022)
- [CAREForMe: Contextual Multi-Armed Bandit Recommendation Framework for Mental Health](https://mlsanity.com/paper/2401.15188/) (2024)
- [Recommender Systems for Good (RS4Good): Survey of Use Cases and a Call to Action for Research that Matters](https://mlsanity.com/paper/2411.16645/) (2024)
- [Using Cognitive Models to Train Warm Start Reinforcement Learning Agents for Human-Computer Interactions](https://mlsanity.com/paper/2103.06160/) (2021)
- [Exploiting Social Graph Networks for Emotion Prediction](https://mlsanity.com/paper/2207.05820/) (2022)
- [Tailored Behavior-Change Messaging for Physical Activity: Integrating Contextual Bandits and Large Language Models](https://mlsanity.com/paper/2506.07275/) (2025)
- [Is Offline Decision Making Possible with Only Few Samples? Reliable Decisions in Data-Starved Bandits via Trust Region Enhancement](https://mlsanity.com/paper/2402.15703/) (2024)
- [Privacy-Preserving UCB Decision Process Verification via zk-SNARKs](https://mlsanity.com/paper/2404.12186/) (2024)
- [Empathic Responding for Digital Interpersonal Emotion Regulation via Content Recommendation](https://mlsanity.com/paper/2408.07704/) (2024)
- [IntOPE: Off-Policy Evaluation in the Presence of Interference](https://mlsanity.com/paper/2408.13484/) (2024)
- [Locally Private Nonparametric Contextual Multi-armed Bandits](https://mlsanity.com/paper/2503.08098/) (2025)

## Abstract (arXiv)

Delivering treatment recommendations via pervasive electronic devices such as mobile phones has the potential to be a viable and scalable treatment medium for long-term health behavior management. But active experimentation of treatment options can be time-consuming, expensive and altogether unethical in some cases. There is a growing interest in methodological approaches that allow an experimenter to learn and evaluate the usefulness of a new treatment strategy before deployment. We present the first development of a treatment recommender system for emotion regulation using real-world historical mobile digital data from n = 114 high socially anxious participants to test the usefulness of new emotion regulation strategies. We explore a number of offline contextual bandits estimators for learning and propose a general framework for learning algorithms. Our experimentation shows that the proposed doubly robust offline learning algorithms performed significantly better than baseline approaches, suggesting that this type of recommender algorithm could improve emotion regulation. Given that emotion regulation is impaired across many mental illnesses and such a recommender algorithm could be scaled up easily, this approach holds potential to increase access to treatment for many people. We also share some insights that allow us to translate contextual bandit models to this complex real-world data, including which contextual features appear to be most important for predicting emotion regulation strategy effectiveness.