# Contextual-Bandit Based Personalized Recommendation with Time-Varying User Interests

arXiv: https://arxiv.org/abs/2003.00359 · HTML: https://mlsanity.com/paper/2003.00359/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Feb 2020.

## Built on by (most cited first, 18 of 18)

- [PAK-UCB Contextual Bandit: An Online Learning Approach to Prompt-Aware Selection of Generative Models and LLMs](https://mlsanity.com/paper/2410.13287/) (2024)
- [HyperBandit: Contextual Bandit with Hypernewtork for Time-Varying User Preferences in Streaming Recommendation](https://mlsanity.com/paper/2308.08497/) (2023)
- [Marginal Density Ratio for Off-Policy Evaluation in Contextual Bandits](https://mlsanity.com/paper/2312.01457/) (2023)
- [Generative Adversarial Reward Learning for Generalized Behavior Tendency Inference](https://mlsanity.com/paper/2105.00822/) (2021)
- [Linear Bandits with Memory: from Rotting to Rising](https://mlsanity.com/paper/2302.08345/) (2023)
- [Learning to Rank For Push Notifications Using Pairwise Expected Regret](https://mlsanity.com/paper/2201.07681/) (2022)
- [Truncated LinUCB for Stochastic Linear Bandits](https://mlsanity.com/paper/2202.11735/) (2022)
- [Maximum entropy exploration in contextual bandits with neural networks and energy based models](https://mlsanity.com/paper/2210.06302/) (2022)
- [Non-stationary Delayed Combinatorial Semi-Bandit with Causally Related Rewards](https://mlsanity.com/paper/2307.09093/) (2023)
- [Enhancing Bandit Algorithms with LLMs for Time-varying User Preferences in Streaming Recommendations](https://mlsanity.com/paper/2602.08067/) (2026)
- [Generalized Reinforcement Learning: Experience Particles, Action Operator, Reinforcement Field, Memory Association, and Decision Concepts](https://mlsanity.com/paper/2208.04822/) (2022)
- [Online Learning with Costly Features in Non-stationary Environments](https://mlsanity.com/paper/2307.09388/) (2023)
- [Diminishing Exploration: A Minimalist Approach to Piecewise Stationary Multi-Armed Bandits](https://mlsanity.com/paper/2410.05734/) (2024)
- [Multi-armed Bandits with Missing Outcome](https://mlsanity.com/paper/2411.05661/) (2024)
- [Learn to Bid as a Price-Maker Wind Power Producer](https://mlsanity.com/paper/2503.16107/) (2025)
- [CoCoB: Adaptive Collaborative Combinatorial Bandits for Online Recommendation](https://mlsanity.com/paper/2505.03840/) (2025)
- [Exploitation Over Exploration: Unmasking the Bias in Linear Bandit Recommender Offline Evaluation](https://mlsanity.com/paper/2507.18756/) (2025)
- [Contextual Scalarisation Thompson Sampling for multi-objective decisions in public media](https://mlsanity.com/paper/2605.31291/) (2026)

## Abstract (arXiv)

A contextual bandit problem is studied in a highly non-stationary environment, which is ubiquitous in various recommender systems due to the time-varying interests of users. Two models with disjoint and hybrid payoffs are considered to characterize the phenomenon that users' preferences towards different items vary differently over time. In the disjoint payoff model, the reward of playing an arm is determined by an arm-specific preference vector, which is piecewise-stationary with asynchronous and distinct changes across different arms. An efficient learning algorithm that is adaptive to abrupt reward changes is proposed and theoretical regret analysis is provided to show that a sublinear scaling of regret in the time length T is achieved. The algorithm is further extended to a more general setting with hybrid payoffs where the reward of playing an arm is determined by both an arm-specific preference vector and a joint coefficient vector shared by all arms. Empirical experiments are conducted on real-world datasets to verify the advantages of the proposed learning algorithms against baseline ones in both settings.