# Power Constrained Nonstationary Bandits with Habituation and Recovery Dynamics

arXiv: https://arxiv.org/abs/2511.02944 · HTML: https://mlsanity.com/paper/2511.02944/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 67 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Builds on (most cited first, 15 of 15)

- [Learning to Optimize Via Posterior Sampling](https://mlsanity.com/paper/1301.2609/) (2013)
- [Fairness in Learning: Classic and Contextual Bandits](https://mlsanity.com/paper/1605.07139/) (2016)
- [Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges](https://mlsanity.com/paper/1507.08025/) (2015)
- [Causal Bandits: Learning Good Interventions via Causal Inference](https://mlsanity.com/paper/1606.03203/) (2016)
- [Efficient Contextual Bandits in Non-stationary Worlds](https://mlsanity.com/paper/1708.01799/) (2017)
- [Power Constrained Bandits](https://mlsanity.com/paper/2004.06230/) (2020)
- [Assessing Time-Varying Causal Effect Moderation in Mobile Health](https://mlsanity.com/paper/1601.00237/) (2016)
- [Non-Stationary Bandits with Habituation and Recovery Dynamics](https://mlsanity.com/paper/1707.08423/) (2017)
- [Non-Stationary Bandit Learning via Predictive Sampling](https://mlsanity.com/paper/2205.01970/) (2022)
- [The Micro-Randomized Trial for Developing Digital Interventions: Experimental Design and Data Analysis Considerations](https://mlsanity.com/paper/2107.03544/) (2021)
- [Policy Optimization for Personalized Interventions in Behavioral Health](https://mlsanity.com/paper/2303.12206/) (2023)
- [Model Based Reinforcement Learning for Personalized Heparin Dosing](https://mlsanity.com/paper/2304.10000/) (2023)
- [Planning a Community Approach to Diabetes Care in Low- and Middle-Income Countries Using Optimization](https://mlsanity.com/paper/2305.06426/) (2023)
- [Non-stationary Bandits with Habituation and Recovery Dynamics and Knapsack Constraints](https://mlsanity.com/paper/2403.17073/) (2024)
- [An Adaptive Control Approach to Treatment Selection for Substance Use Disorders](https://mlsanity.com/paper/2504.01221/) (2025)

## Built on by (most cited first, 1 of 1)

- [Optimizing Digital Therapeutic Interventions: Online Learning under Endogenous Adherence](https://mlsanity.com/paper/2605.24261/) (2026)

## Abstract (arXiv)

A common challenge for decision makers is selecting actions whose rewards are unknown and evolve over time based on prior policies. For instance, repeated use may reduce an action's effectiveness (habituation), while inactivity may restore it (recovery). These nonstationarities are captured by the Reducing or Gaining Unknown Efficacy (ROGUE) bandit framework, which models real-world settings such as behavioral health interventions. While existing algorithms can compute sublinear regret policies to optimize these settings, they may not provide sufficient exploration due to overemphasis on exploitation, limiting the ability to estimate population-level effects. This is a challenge of particular interest in micro-randomized trials (MRTs) that aid researchers in developing just-in-time adaptive interventions that have population-level effects while still providing personalized recommendations to individuals. In this paper, we first develop ROGUE-TS, a Thompson Sampling algorithm tailored to the ROGUE framework, and provide theoretical guarantees of sublinear regret. We then introduce a probability clipping procedure to balance personalization and population-level learning, with quantified trade-off that balances regret and minimum exploration probability. Validation on two MRT datasets concerning physical activity promotion and bipolar disorder treatment shows that our methods both achieve lower regret than existing approaches and maintain high statistical power through the clipping procedure without significantly increasing regret. This enables reliable detection of treatment effects while accounting for individual behavioral dynamics. For researchers designing MRTs, our framework offers practical guidance on balancing personalization with statistical validity.