# Uncertainty of Joint Neural Contextual Bandit

arXiv: https://arxiv.org/abs/2406.02515 · HTML: https://mlsanity.com/paper/2406.02515/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 8 of 8)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Contextual Bandits with Deep Representation and Shallow Exploration](https://mlsanity.com/paper/2012.01780/) (2020)
- [Deep Bayesian Bandits: Exploring in Online Personalized Recommendations](https://mlsanity.com/paper/2008.00727/) (2020)
- [Scalable Neural Contextual Bandit for Recommender Systems](https://mlsanity.com/paper/2306.14834/) (2023)
- [Achieving User-Side Fairness in Contextual Bandits](https://mlsanity.com/paper/2010.12102/) (2020)
- [Recommendation System-based Upper Confidence Bound for Online Advertising](https://mlsanity.com/paper/1909.04190/) (2019)
- [Evaluating Online Bandit Exploration In Large-Scale Recommender System](https://mlsanity.com/paper/2304.02572/) (2023)
- [Two-Stage Neural Contextual Bandits for Personalised News Recommendation](https://mlsanity.com/paper/2206.14648/) (2022)

## Built on by (most cited first, 1 of 1)

- [The Bandit's Blind Spot: The Critical Role of User State Representation in Recommender Systems](https://mlsanity.com/paper/2604.26651/) (2026)

## Abstract (arXiv)

Contextual bandit learning is increasingly favored in modern large-scale recommendation systems. To better utlize the contextual information and available user or item features, the integration of neural networks have been introduced to enhance contextual bandit learning and has triggered significant interest from both academia and industry. However, a major challenge arises when implementing a disjoint neural contextual bandit solution in large-scale recommendation systems, where each item or user may correspond to a separate bandit arm. The huge number of items to recommend poses a significant hurdle for real world production deployment. This paper focuses on a joint neural contextual bandit solution which serves all recommending items in one single model. The output consists of a predicted reward μ, an uncertainty σ and a hyper-parameter α which balances exploitation and exploration, e.g., μ + α σ. The tuning of the parameter α is typically heuristic and complex in practice due to its stochastic nature. To address this challenge, we provide both theoretical analysis and experimental findings regarding the uncertainty σ of the joint neural contextual bandit model. Our analysis reveals that α demonstrates an approximate square root relationship with the size of the last hidden layer F and inverse square root relationship with the amount of training data N, i.e., σ propto √fracFN. The experiments, conducted with real industrial data, align with the theoretical analysis, help understanding model behaviors and assist the hyper-parameter tuning during both offline training and online deployment.