# Modeling Attrition in Recommender Systems with Departing Bandits

arXiv: https://arxiv.org/abs/2203.13423 · HTML: https://mlsanity.com/paper/2203.13423/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 25 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Fairness in Learning: Classic and Contextual Bandits](https://mlsanity.com/paper/1605.07139/) (2016)
- [Achieving Fairness in the Stochastic Multi-armed Bandit Problem](https://mlsanity.com/paper/1907.10516/) (2019)
- [Bayesian Incentive-Compatible Bandit Exploration](https://mlsanity.com/paper/1502.04147/) (2015)
- [Distributed Clustering of Linear Bandits in Peer to Peer Networks](https://mlsanity.com/paper/1604.07706/) (2016)
- [Sequential Transfer in Multi-armed Bandit with Finite Set of Models](https://mlsanity.com/paper/1307.6887/) (2013)
- [Recovering Bandits](https://mlsanity.com/paper/1910.14354/) (2019)
- [Jointly Learning to Recommend and Advertise](https://mlsanity.com/paper/2003.00097/) (2020)
- [Rebounding Bandits for Modeling Satiation Effects](https://mlsanity.com/paper/2011.06741/) (2020)
- [Fast Distributed Bandits for Online Recommendation Systems](https://mlsanity.com/paper/2007.08061/) (2020)
- [Partially Observable Markov Decision Process for Recommender Systems](https://mlsanity.com/paper/1608.07793/) (2016)
- [Fatigue-aware Bandits for Dependent Click Models](https://mlsanity.com/paper/2008.09733/) (2020)
- [Maximizing Cumulative User Engagement in Sequential Recommendation: An Online Optimization Perspective](https://mlsanity.com/paper/2006.04520/) (2020)
- [Fiduciary Bandits](https://mlsanity.com/paper/1905.07043/) (2019)
- [Optimal Algorithm for Bayesian Incentive-Compatible Exploration](https://mlsanity.com/paper/1810.10304/) (2018)
- [Maximum entropy distributions on graphs](https://mlsanity.com/paper/1301.3321/) (2013)

## Built on by (most cited first, 11 of 11)

- [Scalable Neural Contextual Bandit for Recommender Systems](https://mlsanity.com/paper/2306.14834/) (2023)
- [Accounting for AI and Users Shaping One Another: The Role of Mathematical Models](https://mlsanity.com/paper/2404.12366/) (2024)
- [Learning with Exposure Constraints in Recommendation Systems](https://mlsanity.com/paper/2302.01377/) (2023)
- [Optimizing Audio Recommendations for the Long-Term: A Reinforcement Learning Perspective](https://mlsanity.com/paper/2302.03561/) (2023)
- [Modeling Recommender Ecosystems: Research Challenges at the Intersection of Mechanism Design, Reinforcement Learning and Generative Models](https://mlsanity.com/paper/2309.06375/) (2023)
- [Classification Under Strategic Self-Selection](https://mlsanity.com/paper/2402.15274/) (2024)
- [Exploration, Exploitation, and Engagement in Multi-Armed Bandits with Abandonment](https://mlsanity.com/paper/2205.13566/) (2022)
- [Modeling Churn in Recommender Systems with Aggregated Preferences](https://mlsanity.com/paper/2502.18483/) (2025)
- [Algorithmic Content Selection and the Impact of User Disengagement](https://mlsanity.com/paper/2410.13108/) (2024)
- [Churn-Aware Recommendation Planning under Aggregated Preference Feedback](https://mlsanity.com/paper/2507.04513/) (2025)
- [Adaptive Quality-Diversity Trade-offs for Large-Scale Batch Recommendation](https://mlsanity.com/paper/2602.02024/) (2026)

## Abstract (arXiv)

Traditionally, when recommender systems are formalized as multi-armed bandits, the policy of the recommender system influences the rewards accrued, but not the length of interaction. However, in real-world systems, dissatisfied users may depart (and never come back). In this work, we propose a novel multi-armed bandit setup that captures such policy-dependent horizons. Our setup consists of a finite set of user types, and multiple arms with Bernoulli payoffs. Each (user type, arm) tuple corresponds to an (unknown) reward probability. Each user's type is initially unknown and can only be inferred through their response to recommendations. Moreover, if a user is dissatisfied with their recommendation, they might depart the system. We first address the case where all users share the same type, demonstrating that a recent UCB-based algorithm is optimal. We then move forward to the more challenging case, where users are divided among two types. While naive approaches cannot handle this setting, we provide an efficient learning algorithm that achieves tildeO(√T) regret, where T is the number of users.