# Contextual Bandits with Sparse Data in Web setting

arXiv: https://arxiv.org/abs/2105.02873 · HTML: https://mlsanity.com/paper/2105.02873/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 26 references that are in the map.
- First posted on arXiv 6 May 2021.

## Builds on (most cited first, 9 of 9)

- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [Deep Reinforcement Learning based Recommendation with Explicit User-Item Interactions Modeling](https://mlsanity.com/paper/1810.12027/) (2018)
- [Do Offline Metrics Predict Online Performance in Recommender Systems?](https://mlsanity.com/paper/2011.07931/) (2020)
- [Safe Exploration for Optimizing Contextual Bandits](https://mlsanity.com/paper/2002.00467/) (2020)
- [Bayesian Meta-Prior Learning Using Empirical Bayes](https://mlsanity.com/paper/2002.01129/) (2020)
- [Exploration in Online Advertising Systems with Deep Uncertainty-Aware Learning](https://mlsanity.com/paper/2012.02298/) (2020)
- [Contextual Hybrid Session-based News Recommendation with Recurrent Neural Networks](https://mlsanity.com/paper/1904.10367/) (2019)
- [Handling Cold-Start Collaborative Filtering with Reinforcement Learning](https://mlsanity.com/paper/1806.06192/) (2018)
- [Online Learning and Decision-Making under Generalized Linear Model with High-Dimensional Data](https://mlsanity.com/paper/1812.02962/) (2018)

## Abstract (arXiv)

This paper is a scoping study to identify current methods used in handling sparse data with contextual bandits in web settings. The area is highly current and state of the art methods are identified. The years 2017-2020 are investigated, and 19 method articles are identified, and two review articles. Five categories of methods are described, making it easy to choose how to address sparse data using contextual bandits with a method available for modification in the specific setting of concern. In addition, each method has multiple techniques to choose from for future evaluation. The problem areas are also mentioned that each article covers. An overall updated understanding of sparse data problems using contextual bandits in web settings is given. The identified methods are policy evaluation (off-line and on-line) , hybrid-method, model representation (clusters and deep neural networks), dimensionality reduction, and simulation.