# Deep reinforcement learning for search, recommendation, and online advertising: a survey

arXiv: https://arxiv.org/abs/1812.07127 · HTML: https://mlsanity.com/paper/1812.07127/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2018.

## Built on by (most cited first, 23 of 23)

- [Click-Through Rate Prediction in Online Advertising: A Literature Review](https://mlsanity.com/paper/2202.10462/) (2022)
- [Multi-Task Recommendations with Reinforcement Learning](https://mlsanity.com/paper/2302.03328/) (2023)
- [Generative Auto-Bidding with Value-Guided Explorations](https://mlsanity.com/paper/2504.14587/) (2025)
- [Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents](https://mlsanity.com/paper/2602.10226/) (2026)
- [Impatient Bandits: Optimizing Recommendations for the Long-Term Without Delay](https://mlsanity.com/paper/2307.09943/) (2023)
- [Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation](https://mlsanity.com/paper/2109.08331/) (2021)
- [DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agent](https://mlsanity.com/paper/2603.01152/) (2026)
- [Debiased Recommendation with Noisy Feedback](https://mlsanity.com/paper/2406.17182/) (2024)
- [Optimizing Long-term Value for Auction-Based Recommender Systems via On-Policy Reinforcement Learning](https://mlsanity.com/paper/2305.13747/) (2023)
- [Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learning](https://mlsanity.com/paper/2409.14557/) (2024)
- [RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systems](https://mlsanity.com/paper/2508.00201/) (2025)
- [AdCraft: An Advanced Reinforcement Learning Benchmark Environment for Search Engine Marketing Optimization](https://mlsanity.com/paper/2306.11971/) (2023)
- [CDR: Conservative Doubly Robust Learning for Debiased Recommendation](https://mlsanity.com/paper/2308.08461/) (2023)
- [Learning to Model Diverse Driving Behaviors in Highly Interactive Autonomous Driving Scenarios with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2402.13481/) (2024)
- [Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems](https://mlsanity.com/paper/2506.23090/) (2025)
- [Sequential Search with Off-Policy Reinforcement Learning](https://mlsanity.com/paper/2202.00245/) (2022)
- [AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation](https://mlsanity.com/paper/2308.06965/) (2023)
- [Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendation](https://mlsanity.com/paper/2310.16566/) (2023)
- [Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest](https://mlsanity.com/paper/2509.05292/) (2025)
- [Bid2X: Revealing Dynamics of Bidding Environment in Online Advertising from A Foundation Model Lens](https://mlsanity.com/paper/2510.23410/) (2025)
- [C2:Cross learning module enhanced decision transformer with Constraint-aware loss for auto-bidding](https://mlsanity.com/paper/2601.20257/) (2026)
- [A Deep Reinforcement Learning Approach for Interactive Search with Sentence-level Feedback](https://mlsanity.com/paper/2310.03043/) (2023)
- [A Production-Ready RL Framework for Personalized Utility Tuning with Pareto Sweeping in Pinterest Recommender Systems](https://mlsanity.com/paper/2605.16344/) (2026)

## Abstract (arXiv)

Search, recommendation, and online advertising are the three most important information-providing mechanisms on the web. These information seeking techniques, satisfying users' information needs by suggesting users personalized objects (information or services) at the appropriate time and place, play a crucial role in mitigating the information overload problem. With recent great advances in deep reinforcement learning (DRL), there have been increasing interests in developing DRL based information seeking techniques. These DRL based techniques have two key advantages -- (1) they are able to continuously update information seeking strategies according to users' real-time feedback, and (2) they can maximize the expected cumulative long-term reward from users where reward has different definitions according to information seeking applications such as click-through rate, revenue, user satisfaction and engagement. In this paper, we give an overview of deep reinforcement learning for search, recommendation, and online advertising from methodologies to applications, review representative algorithms, and discuss some appealing research directions.