# Generator-Mediated Bandits: Thompson Sampling for GenAI-Powered Adaptive Interventions

arXiv: https://arxiv.org/abs/2505.16311 · HTML: https://mlsanity.com/paper/2505.16311/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Causal Bandits: Learning Good Interventions via Causal Inference](https://mlsanity.com/paper/1606.03203/) (2016)
- [Deep Reinforcement Learning with a Natural Language Action Space](https://mlsanity.com/paper/1511.04636/) (2015)
- [Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity](https://mlsanity.com/paper/1909.03539/) (2019)
- [Ensemble Sampling](https://mlsanity.com/paper/1705.07347/) (2017)
- [How to Make Causal Inferences Using Texts](https://mlsanity.com/paper/1802.02163/) (2018)
- [Nearly Minimax-Optimal Regret for Linearly Parameterized Bandits](https://mlsanity.com/paper/1904.00242/) (2019)
- [Causal Effects of Linguistic Properties](https://mlsanity.com/paper/2010.12919/) (2020)
- [Assessing Time-Varying Causal Effect Moderation in Mobile Health](https://mlsanity.com/paper/1601.00237/) (2016)
- [Contextual Bandits with Latent Confounders: An NMF Approach](https://mlsanity.com/paper/1606.00119/) (2016)
- [Pythae: Unifying Generative Autoencoders in Python -- A Benchmarking Use Case](https://mlsanity.com/paper/2206.08309/) (2022)
- [IntelligentPooling: Practical Thompson Sampling for mHealth](https://mlsanity.com/paper/2008.01571/) (2020)
- [Causal Bandits with Propagating Inference](https://mlsanity.com/paper/1806.02252/) (2018)
- [Causal Inference with Generative Artificial Intelligence: Application to Texts as Treatments](https://mlsanity.com/paper/2410.00903/) (2024)
- [A Causal Bandit Approach to Learning Good Atomic Interventions in Presence of Unobserved Confounders](https://mlsanity.com/paper/2107.02772/) (2021)
- [Instrument-Armed Bandits](https://mlsanity.com/paper/1705.07377/) (2017)
- [Personalized Policy Learning using Longitudinal Mobile Health Data](https://mlsanity.com/paper/2001.03258/) (2020)
- [Combinatorial Pure Exploration of Causal Bandits](https://mlsanity.com/paper/2206.07883/) (2022)
- [Stochastic Online Instrumental Variable Regression: Regrets for Endogeneity and Bandit Feedback](https://mlsanity.com/paper/2302.09357/) (2023)
- [RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions](https://mlsanity.com/paper/2312.06403/) (2023)
- [Information Capacity Regret Bounds for Bandits with Mediator Feedback](https://mlsanity.com/paper/2402.10282/) (2024)
- [Thompson Sampling for Noncompliant Bandits](https://mlsanity.com/paper/1812.00856/) (2018)
- [Causal Bandits: The Pareto Optimal Frontier of Adaptivity, a Reduction to Linear Bandits, and Limitations around Unknown Marginals](https://mlsanity.com/paper/2407.00950/) (2024)

## Built on by (most cited first, 1 of 1)

- [Tailored Behavior-Change Messaging for Physical Activity: Integrating Contextual Bandits and Large Language Models](https://mlsanity.com/paper/2506.07275/) (2025)

## Abstract (arXiv)

Recent advances in generative artificial intelligence (GenAI) models have enabled the generation of personalized content that adapts to up-to-date user context. While personalized decision systems are often modeled using bandit formulations, the integration of GenAI introduces new structure into otherwise classical sequential learning problems. In GenAI-powered interventions, the agent selects a query, but the environment experiences a stochastic response drawn from the generative model. Standard bandit methods do not explicitly account for this structure, where actions influence rewards only through stochastic, observed treatments. We introduce generator-mediated bandit-Thompson sampling (GAMBITTS), a bandit approach designed for this action/treatment split, using mobile health interventions with large language model-generated text as a motivating case study. GAMBITTS explicitly models both the treatment and reward generation processes, using information in the delivered treatment to accelerate policy learning relative to standard methods. We establish regret bounds for GAMBITTS by decomposing sources of uncertainty in treatment and reward, identifying conditions where it achieves stronger guarantees than standard bandit approaches. In simulation studies, GAMBITTS consistently outperforms conventional algorithms by leveraging observed treatments to more accurately estimate expected rewards.