# Exploration in Online Advertising Systems with Deep Uncertainty-Aware Learning

arXiv: https://arxiv.org/abs/2012.02298 · HTML: https://mlsanity.com/paper/2012.02298/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2020.

## Built on by (most cited first, 12 of 12)

- [Efficient Online Bayesian Inference for Neural Bandits](https://mlsanity.com/paper/2112.00195/) (2021)
- [ROI-Constrained Bidding via Curriculum-Guided Bayesian Reinforcement Learning](https://mlsanity.com/paper/2206.05240/) (2022)
- [Adversarial Constrained Bidding via Minimax Regret Optimization with Causality-Aware Reinforcement Learning](https://mlsanity.com/paper/2306.07106/) (2023)
- [Hybrid Transfer in Deep Reinforcement Learning for Ads Allocation](https://mlsanity.com/paper/2204.11589/) (2022)
- [Adversarial Gradient Driven Exploration for Deep Click-Through Rate Prediction](https://mlsanity.com/paper/2112.11136/) (2021)
- [Deep Ensemble Shape Calibration: Multi-Field Post-hoc Calibration in Online Advertising](https://mlsanity.com/paper/2401.09507/) (2024)
- [Contextual Bandits with Sparse Data in Web setting](https://mlsanity.com/paper/2105.02873/) (2021)
- [Interactive Graph Convolutional Filtering](https://mlsanity.com/paper/2309.01453/) (2023)
- [Epsilon non-Greedy: A Bandit Approach for Unbiased Recommendation via Uniform Data](https://mlsanity.com/paper/2310.04855/) (2023)
- [Deep Uncertainty-Based Explore for Index Construction and Retrieval in Recommendation System](https://mlsanity.com/paper/2408.00799/) (2024)
- [MBD: A Model-Based Debiasing Framework Across User, Content, and Model Dimensions](https://mlsanity.com/paper/2603.14422/) (2026)
- [LeAP: Learnable Adaptive Permutation for Feature Selection in Heterogeneous and Sparse Recommender Systems](https://mlsanity.com/paper/2606.01111/) (2026)

## Abstract (arXiv)

Modern online advertising systems inevitably rely on personalization methods, such as click-through rate (CTR) prediction. Recent progress in CTR prediction enjoys the rich representation capabilities of deep learning and achieves great success in large-scale industrial applications. However, these methods can suffer from lack of exploration. Another line of prior work addresses the exploration-exploitation trade-off problem with contextual bandit methods, which are recently less studied in the industry due to the difficulty in extending their flexibility with deep models. In this paper, we propose a novel Deep Uncertainty-Aware Learning (DUAL) method to learn CTR models based on Gaussian processes, which can provide predictive uncertainty estimations while maintaining the flexibility of deep neural networks. DUAL can be easily implemented on existing models and deployed in real-time systems with minimal extra computational overhead. By linking the predictive uncertainty estimation ability of DUAL to well-known bandit algorithms, we further present DUAL-based Ad-ranking strategies to boost up long-term utilities such as the social welfare in advertising systems. Experimental results on several public datasets demonstrate the effectiveness of our methods. Remarkably, an online A/B test deployed in the Alibaba display advertising platform shows an 8.2% social welfare improvement and an 8.0% revenue lift.