# Neural Contextual Bandits for Personalized Recommendation

arXiv: https://arxiv.org/abs/2312.14037 · HTML: https://mlsanity.com/paper/2312.14037/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 21 Dec 2023.

## Builds on (most cited first, 25 of 25)

- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oracles](https://mlsanity.com/paper/2002.04926/) (2020)
- [Finite-Time Analysis of Kernelised Contextual Bandits](https://mlsanity.com/paper/1309.6869/) (2013)
- [Neural Thompson Sampling](https://mlsanity.com/paper/2010.00827/) (2020)
- [Online Clustering of Bandits](https://mlsanity.com/paper/1401.8257/) (2014)
- [Distributed Clustering of Linear Bandits in Peer to Peer Networks](https://mlsanity.com/paper/1604.07706/) (2016)
- [Multi-Task Learning for Contextual Bandits](https://mlsanity.com/paper/1705.08618/) (2017)
- [On Context-Dependent Clustering of Bandits](https://mlsanity.com/paper/1608.03544/) (2016)
- [EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits](https://mlsanity.com/paper/2110.03177/) (2021)
- [Neural Contextual Bandits without Regret](https://mlsanity.com/paper/2107.03144/) (2021)
- [Improved Algorithm on Online Clustering of Bandits](https://mlsanity.com/paper/1902.09162/) (2019)
- [Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation](https://mlsanity.com/paper/2307.04571/) (2023)
- [Local Clustering in Contextual Multi-Armed Bandits](https://mlsanity.com/paper/2103.00063/) (2021)
- [Multi-facet Contextual Bandits: A Neural Network Perspective](https://mlsanity.com/paper/2106.03039/) (2021)
- [Graph Neural Bandits](https://mlsanity.com/paper/2308.10808/) (2023)
- [Learning Neural Contextual Bandits Through Perturbed Rewards](https://mlsanity.com/paper/2201.09910/) (2022)
- [Impatient Bandits: Optimizing Recommendations for the Long-Term Without Delay](https://mlsanity.com/paper/2307.09943/) (2023)
- [Generic Outlier Detection in Multi-Armed Bandit](https://mlsanity.com/paper/2007.07293/) (2020)
- [Improved Algorithms for Neural Active Learning](https://mlsanity.com/paper/2210.00423/) (2022)
- [ResAct: Reinforcing Long-term Engagement in Sequential Recommendation with Residual Actor](https://mlsanity.com/paper/2206.02620/) (2022)
- [Neural Bandit with Arm Group Graph](https://mlsanity.com/paper/2206.03644/) (2022)
- [Neural Collaborative Filtering Bandits via Meta Learning](https://mlsanity.com/paper/2201.13395/) (2022)
- [Neural Exploitation and Exploration of Contextual Bandits](https://mlsanity.com/paper/2305.03784/) (2023)
- [Contextual Bandits with Online Neural Regression](https://mlsanity.com/paper/2312.07145/) (2023)

## Built on by (most cited first, 10 of 10)

- [Discrete-state Continuous-time Diffusion for Graph Generation](https://mlsanity.com/paper/2405.11416/) (2024)
- [Meta Clustering of Neural Bandits](https://mlsanity.com/paper/2408.05586/) (2024)
- [OLIVIA: Online Learning via Inference-time Action Adaptation for Decision Making in LLM ReAct Agents](https://mlsanity.com/paper/2605.11169/) (2026)
- [Constrained Online Decision-Making: A Unified Framework](https://mlsanity.com/paper/2505.07101/) (2025)
- [Multi-armed Bandits with Missing Outcome](https://mlsanity.com/paper/2411.05661/) (2024)
- [Causal Predictive Optimization and Generation for Business AI](https://mlsanity.com/paper/2505.09847/) (2025)
- [Co-Exploration and Co-Exploitation via Shared Structure in Multi-Task Bandits](https://mlsanity.com/paper/2512.12693/) (2025)
- [Adapting Actively on the Fly: Relevance-Guided Online Meta-Learning with Latent Concepts for Geospatial Discovery](https://mlsanity.com/paper/2602.17605/) (2026)
- [Inverse Contextual Bandits without Rewards: Learning from a Non-Stationary Learner via Suffix Imitation](https://mlsanity.com/paper/2603.03778/) (2026)
- [A Practical Algorithm for Feature-Rich, Non-Stationary Bandit Problems](https://mlsanity.com/paper/2603.16755/) (2026)

## Abstract (arXiv)

In the dynamic landscape of online businesses, recommender systems are pivotal in enhancing user experiences. While traditional approaches have relied on static supervised learning, the quest for adaptive, user-centric recommendations has led to the emergence of the formulation of contextual bandits. This tutorial investigates the contextual bandits as a powerful framework for personalized recommendations. We delve into the challenges, advanced algorithms and theories, collaborative strategies, and open challenges and future prospects within this field. Different from existing related tutorials, (1) we focus on the exploration perspective of contextual bandits to alleviate the ``Matthew Effect'' in the recommender systems, i.e., the rich get richer and the poor get poorer, concerning the popularity of items; (2) in addition to the conventional linear contextual bandits, we will also dedicated to neural contextual bandits which have emerged as an important branch in recent years, to investigate how neural networks benefit contextual bandits for personalized recommendation both empirically and theoretically; (3) we will cover the latest topic, collaborative neural contextual bandits, to incorporate both user heterogeneity and user correlations customized for recommender system; (4) we will provide and discuss the new emerging challenges and open questions for neural contextual bandits with applications in the personalized recommendation, especially for large neural models.