mlsanityData to 14 Oct 2026

Neural Contextual Bandits with Deep Representation and Shallow Exploration

arXiv 2012.01780 ↗First posted 3 Dec 2020cs.LG stat.MLcited by 71

Not on the map. Why?

Abstract (arXiv)

We study a general class of contextual bandits, where each context-action pair is associated with a raw feature vector, but the reward generating function is unknown. We propose a novel learning algorithm that transforms the raw feature vector using the last hidden layer of a deep ReLU neural network (deep representation learning), and uses an upper confidence bound (UCB) approach to explore in the last linear layer (shallow exploration). We prove that under standard assumptions, our proposed algorithm achieves tildeO(√T) finite-time regret, where T is the learning time horizon. Compared with existing neural contextual bandit algorithms, our approach is computationally much more efficient since it only needs to explore in the last layer of the deep neural network.

Key facts. Cited by 71 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 10; 2023: 12; 2024: 22; 2025: 12; 2026: 7.

71 papers.

Sort:

By area:

  1. 2021EE-Net: Exploitation-Exploration Neural Networks in Contextual Banditscited by 44
  2. 2022Contextual Bandits with Large Action Spaces: Made Practicalcited by 33
  3. 2024Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimationcited by 33
  4. 2021Offline Neural Contextual Bandits: Pessimism, Optimization and Generalizationcited by 25
  5. 2022Sample-Then-Optimize Batch Neural Thompson Samplingcited by 25
  6. 2021Batched Neural Banditscited by 22
  7. 2022Langevin Monte Carlo for Contextual Banditscited by 22
  8. 2025Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuningcited by 21
  9. 2022Learning Neural Contextual Bandits Through Perturbed Rewardscited by 17
  10. 2022Federated Neural Banditscited by 17
  11. 2021Optimal Gradient-based Algorithms for Non-concave Bandit Optimizationcited by 15
  12. 2021Online Limited Memory Neural-Linear Bandits with Likelihood Matchingcited by 14
38 more
  1. 2021Deep Exploration for Recommendation Systemscited by 14
  2. 2024PageRank Bandits for Link Predictioncited by 14
  3. 2026Real-Time Aligned Reward Model beyond Semanticscited by 13
  4. 2023Scalable Neural Contextual Bandit for Recommender Systemscited by 12
  5. 2022Hierarchical Conversational Preference Elicitation with Bandit Feedbackcited by 10
  6. 2023Pearl: A Production-ready Reinforcement Learning Agentcited by 9
  7. 2023Doubly High-Dimensional Contextual Bandits: An Interpretable Model for Joint Assortment-Pricingcited by 8
  8. 2023Evaluating Online Bandit Exploration In Large-Scale Recommender Systemcited by 7
  9. 2023Neural Exploitation and Exploration of Contextual Banditscited by 7
  10. 2024Meta Clustering of Neural Banditscited by 7
  11. 2022Neural Design for Genetic Perturbation Experimentscited by 6
  12. 2023VITS : Variational Inference Thompson Sampling for contextual banditscited by 6
  13. 2024Active Preference Learning for Ordering Items In- and Out-of-samplecited by 6
  14. 2025Neural Logistic Banditscited by 6
  15. 2022Scalable Representation Learning in Linear Contextual Bandits with Constant Regret Guaranteescited by 5
  16. 2023Representation-Driven Reinforcement Learningcited by 4
  17. 2025Active Human Feedback Collection via Neural Contextual Dueling Banditscited by 4
  18. 2025Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdowncited by 4
  19. 2021Neural Pseudo-Label Optimism for the Bank Loan Problemcited by 3
  20. 2023Uncertainty-Aware Instance Reweighting for Off-Policy Learningcited by 3
  21. 2024Tree Search-Based Evolutionary Bandits for Protein Sequence Optimizationcited by 3
  22. 2024Randomized Confidence Bounds for Stochastic Partial Monitoringcited by 3
  23. 2024Efficient Online Set-valued Classification with Bandit Feedbackcited by 3
  24. 2024Variance-Aware Linear UCB with Deep Representation for Neural Contextual Banditscited by 3
  25. 2025Online Clustering of Dueling Banditscited by 3
  26. 2022Reward-Biased Maximum Likelihood Estimation for Neural Contextual Banditscited by 2
  27. 2022Two-Stage Neural Contextual Bandits for Personalised News Recommendationcited by 2
  28. 2023Neural-BO: A Black-box Optimization Algorithm using Deep Neural Networkscited by 2
  29. 2024Stochastic Bandits with ReLU Neural Networkscited by 2
  30. 2024A Contextual Combinatorial Bandit Approach to Negotiationcited by 2
  31. 2024Adaptive Behavioral AI: Reinforcement Learning to Enhance Pharmacy Servicescited by 2
  32. 2025Neural Variance-aware Dueling Bandits with Deep Representation and Shallow Explorationcited by 2
  33. 2025Provable Anytime Ensemble Sampling Algorithms in Nonlinear Contextual Banditscited by 2
  34. 2021Empirical analysis of representation learning and exploration in neural kernel banditscited by 1
  35. 2023Non-Stationary Contextual Bandit Learning via Neural Predictive Ensemble Samplingcited by 1
  36. 2023Risk-Aware Continuous Control with Neural Contextual Banditscited by 1
  37. 2024ε-Neural Thompson Sampling of Deep Brain Stimulation for Parkinson Disease Treatmentcited by 1
  38. 2024Neural Active Learning Meets the Partial Monitoring Frameworkcited by 1

50 most cited of 71 All 71 (JSON)