# Modeling Mobile Health Users as Reinforcement Learning Agents

arXiv: https://arxiv.org/abs/2212.00863 · HTML: https://mlsanity.com/paper/2212.00863/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 8 of 8)

- [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020)
- [On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inference](https://mlsanity.com/paper/1906.09624/) (2019)
- [Learning the Preferences of Ignorant, Inconsistent Agents](https://mlsanity.com/paper/1512.05832/) (2015)
- [Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior](https://mlsanity.com/paper/1805.08010/) (2018)
- [The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Models](https://mlsanity.com/paper/2204.10759/) (2022)
- [Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics](https://mlsanity.com/paper/1604.03912/) (2016)
- [MIRROR: Differentiable Deep Social Projection for Assistive Human-Robot Communication](https://mlsanity.com/paper/2203.02877/) (2022)
- [Robots that Take Advantage of Human Trust](https://mlsanity.com/paper/1909.05777/) (2019)

## Built on by (most cited first, 2 of 2)

- [Graph-enabled Reinforcement Learning for Time Series Forecasting with Adaptive Intelligence](https://mlsanity.com/paper/2309.10186/) (2023)
- [Discovering User Types: Mapping User Traits by Task-Specific Behaviors in Reinforcement Learning](https://mlsanity.com/paper/2307.08169/) (2023)

## Abstract (arXiv)

Mobile health (mHealth) technologies empower patients to adopt/maintain healthy behaviors in their daily lives, by providing interventions (e.g. push notifications) tailored to the user's needs. In these settings, without intervention, human decision making may be impaired (e.g. valuing near term pleasure over own long term goals). In this work, we formalize this relationship with a framework in which the user optimizes a (potentially impaired) Markov Decision Process (MDP) and the mHealth agent intervenes on the user's MDP parameters. We show that different types of impairments imply different types of optimal intervention. We also provide analytical and empirical explorations of these differences.