# Recommendation System-based Upper Confidence Bound for Online Advertising

arXiv: https://arxiv.org/abs/1909.04190 · HTML: https://mlsanity.com/paper/1909.04190/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2019.

## Built on by (most cited first, 7 of 7)

- [Deep Exploration for Recommendation Systems](https://mlsanity.com/paper/2109.12509/) (2021)
- [Evaluating Online Bandit Exploration In Large-Scale Recommender System](https://mlsanity.com/paper/2304.02572/) (2023)
- [Adversarial Gradient Driven Exploration for Deep Click-Through Rate Prediction](https://mlsanity.com/paper/2112.11136/) (2021)
- [Multi-User Reinforcement Learning with Low Rank Rewards](https://mlsanity.com/paper/2210.05355/) (2022)
- [Uncertainty of Joint Neural Contextual Bandit](https://mlsanity.com/paper/2406.02515/) (2024)
- [Thompson Sampling on Asymmetric α-Stable Bandits](https://mlsanity.com/paper/2203.10214/) (2022)
- [Data Acquisition for Improving Model Fairness using Reinforcement Learning](https://mlsanity.com/paper/2412.03009/) (2024)

## Abstract (arXiv)

In this paper, the method UCB-RS, which resorts to recommendation system (RS) for enhancing the upper-confidence bound algorithm UCB, is presented. The proposed method is used for dealing with non-stationary and large-state spaces multi-armed bandit problems. The proposed method has been targeted to the problem of the product recommendation in the online advertising. Through extensive testing with RecoGym, an OpenAI Gym-based reinforcement learning environment for the product recommendation in online advertising, the proposed method outperforms the widespread reinforcement learning schemes such as ε-Greedy, Upper Confidence (UCB1) and Exponential Weights for Exploration and Exploitation (EXP3).