mlsanityData to 7 Oct 2026

DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systems

arXiv 1909.03602 ↗First posted 9 Sep 2019cs.IR cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

With the recent prevalence of Reinforcement Learning (RL), there have been tremendous interests in utilizing RL for online advertising in recommendation platforms (e.g., e-commerce and news feed sites). However, most RL-based advertising algorithms focus on optimizing ads' revenue while ignoring the possible negative influence of ads on user experience of recommended items (products, articles and videos). Developing an optimal advertising algorithm in recommendations faces immense challenges because interpolating ads improperly or too frequently may decrease user experience, while interpolating fewer ads will reduce the advertising revenue. Thus, in this paper, we propose a novel advertising strategy for the rec/ads trade-off. To be specific, we develop an RL-based framework that can continuously update its advertising strategies and maximize reward in the long run. Given a recommendation list, we design a novel Deep Q-network architecture that can determine three internally related tasks jointly, i.e., (i) whether to interpolate an ad or not in the recommendation list, and if yes, (ii) the optimal ad and (iii) the optimal location to interpolate. The experimental results based on real-world data demonstrate the effectiveness of the proposed framework.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 6; 2023: 10; 2024: 7; 2025: 9; 2026: 7.

44 papers.

Sort:

By area:

  1. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  2. 2025Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learningcited by 68
  3. 2021AutoLoss: Automated Loss Function Search in Recommendationscited by 31
  4. 2023Multi-Task Recommendations with Reinforcement Learningcited by 26
  5. 2025Generative Auto-Bidding with Value-Guided Explorationscited by 23
  6. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 19
  7. 2021Cross DQN: Cross Deep Q Network for Ads Allocation in Feedcited by 18
  8. 2021Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulationcited by 16
  9. 2023A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviorscited by 13
  10. 2022Deep Page-Level Interest Network in Reinforcement Learning for Ads Allocationcited by 8
  11. 2022Hierarchically Constrained Adaptive Ad Exposure in Feedscited by 6
  12. 2024Deep Automated Mechanism Design for Integrating Ad Auction and Allocation in Feedcited by 6
32 more
  1. 2022Hybrid Transfer in Deep Reinforcement Learning for Ads Allocationcited by 5
  2. 2021Benchmarking Perturbation-based Saliency Maps for Explaining Atari Agentscited by 4
  3. 2024DNS-Rec: Data-aware Neural Architecture Search for Recommender Systemscited by 4
  4. 2022Learning List-wise Representation in Reinforcement Learning for Ads Allocation with Multiple Auxiliary Taskscited by 3
  5. 2023RL-MPCA: A Reinforcement Learning Based Multi-Phase Computation Allocation Approach for Recommender Systemscited by 3
  6. 2023Provably Efficient Algorithm for Nonstationary Low-Rank MDPscited by 2
  7. 2025Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systemscited by 2
  8. 2022Sequential Search with Off-Policy Reinforcement Learningcited by 1
  9. 2023XDQN: Inherently Interpretable DQN through Mimickingcited by 1
  10. 2023MDDL: A Framework for Reinforcement Learning-based Position Allocation in Multi-Channel Feedcited by 1
  11. 2023AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendationcited by 1
  12. 2023Model-free Reinforcement Learning with Stochastic Reward Stabilization for Recommender Systemscited by 1
  13. 2023Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendationcited by 1
  14. 2023WellFactor: Patient Profiling using Integrative Embedding of Healthcare Datacited by 1
  15. 2024Analyzing and Bridging the Gap between Maximizing Total Reward and Discounted Reward in Deep Reinforcement Learningcited by 1
  16. 2024FedSlate:A Federated Deep Reinforcement Learning Recommender Systemcited by 1
  17. 2024Ads Supply Personalization via Doubly Robust Learningcited by 1
  18. 2024Guided Diffusion-based Counterfactual Augmentation for Robust Session-based Recommendationcited by 1
  19. 2025Enhancing Offline Model-Based RL via Active Model Selection: A Bayesian Optimization Perspectivecited by 1
  20. 2025Predictive Response Optimization: Using Reinforcement Learning to Fight Online Social Network Abusecited by 1
  21. 2025Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterestcited by 1
  22. 2026Inference Time Policy Optimization for Offline RL with Differentiable World Modelscited by 1
  23. 2024MODRL-TA:A Multi-Objective Deep Reinforcement Learning Framework for Traffic Allocation in E-Commerce Searchcited by 0
  24. 2025BLIA: Detect model memorization in binary classification model through passive Label Inference attackcited by 0
  25. 2025KLAN: Kuaishou Landing-page Adaptive Navigatorcited by 0
  26. 2025MaRCA: Multi-Agent Reinforcement Learning for Dynamic Computation Allocation in Large-Scale Recommender Systemscited by 0
  27. 2026C2:Cross learning module enhanced decision transformer with Constraint-aware loss for auto-biddingcited by 0
  28. 2026One Model, Two Markets: Bid-Aware Generative Recommendationcited by 0
  29. 2026FreeScale: Distributed Training for Sequence Recommendation Models with Minimal Scaling Costcited by 0
  30. 2026A Production-Ready RL Framework for Personalized Utility Tuning with Pareto Sweeping in Pinterest Recommender Systemscited by 0
  31. 2026An Interpretable CF-RL-TOPSIS Fusion Model for Skills-Aware Talent Recommendationcited by 0
  32. 2026Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Studycited by 0