# Sequential Choice Bandits with Feedback for Personalizing users' experience

arXiv: https://arxiv.org/abs/2101.01572 · HTML: https://mlsanity.com/paper/2101.01572/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 5 Jan 2021.

## Builds on (most cited first, 11 of 11)

- [Minimax Regret Bounds for Reinforcement Learning](https://mlsanity.com/paper/1703.05449/) (2017)
- [Is Q-learning Provably Efficient?](https://mlsanity.com/paper/1807.03765/) (2018)
- [Cascading Bandits: Learning to Rank in the Cascade Model](https://mlsanity.com/paper/1502.02763/) (2015)
- [What Doubling Tricks Can and Can't Do for Multi-Armed Bandits](https://mlsanity.com/paper/1803.06971/) (2018)
- [On Explore-Then-Commit Strategies](https://mlsanity.com/paper/1605.08988/) (2016)
- [Unifying the stochastic and the adversarial Bandits with Knapsack](https://mlsanity.com/paper/1811.12253/) (2018)
- [Floyd-Warshall Reinforcement Learning: Learning from Past Experiences to Reach New Goals](https://mlsanity.com/paper/1809.09318/) (2018)
- [Online learning with feedback graphs and switching costs](https://mlsanity.com/paper/1810.09666/) (2018)
- [Dynamic Learning of Sequential Choice Bandit Problem under Marketing Fatigue](https://mlsanity.com/paper/1903.08193/) (2019)
- [Revenue Maximization and Learning in Products Ranking](https://mlsanity.com/paper/2012.03800/) (2020)
- [Learning with Abandonment](https://mlsanity.com/paper/1802.08718/) (2018)

## Built on by (most cited first, 2 of 2)

- [Diversified Recommendations for Agents with Adaptive Preferences](https://mlsanity.com/paper/2210.07773/) (2022)
- [Online Recommendations for Agents with Discounted Adaptive Preferences](https://mlsanity.com/paper/2302.06014/) (2023)

## Abstract (arXiv)

In this work, we study sequential choice bandits with feedback. We propose bandit algorithms for a platform that personalizes users' experience to maximize its rewards. For each action directed to a given user, the platform is given a positive reward, which is a non-decreasing function of the action, if this action is below the user's threshold. Users are equipped with a patience budget, and actions that are above the threshold decrease the user's patience. When all patience is lost, the user abandons the platform. The platform attempts to learn the thresholds of the users in order to maximize its rewards, based on two different feedback models describing the information pattern available to the platform at each action. We define a notion of regret by determining the best action to be taken when the platform knows that the user's threshold is in a given interval. We then propose bandit algorithms for the two feedback models and show that upper and lower bounds on the regret are of the order of tildeO(N2/3) and tildeOmega(N2/3), respectively, where N is the total number of users. Finally, we show that the waiting time of any user before receiving a personalized experience is uniform in N.