# Jointly Learning to Recommend and Advertise

arXiv: https://arxiv.org/abs/2003.00097 · HTML: https://mlsanity.com/paper/2003.00097/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2020.

## Built on by (most cited first, 27 of 27)

- [AutoLoss: Automated Loss Function Search in Recommendations](https://mlsanity.com/paper/2106.06713/) (2021)
- [A Survey of Personalization: From RAG to Agent](https://mlsanity.com/paper/2504.10147/) (2025)
- [Generative Auto-Bidding with Value-Guided Explorations](https://mlsanity.com/paper/2504.14587/) (2025)
- [A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions](https://mlsanity.com/paper/2109.03540/) (2021)
- [Cross DQN: Cross Deep Q Network for Ads Allocation in Feed](https://mlsanity.com/paper/2109.04353/) (2021)
- [A General Offline Reinforcement Learning Framework for Interactive Recommendation](https://mlsanity.com/paper/2310.00678/) (2023)
- [User Response Prediction in Online Advertising](https://mlsanity.com/paper/2101.02342/) (2021)
- [Modeling Attrition in Recommender Systems with Departing Bandits](https://mlsanity.com/paper/2203.13423/) (2022)
- [Recommendation Fairness: From Static to Dynamic](https://mlsanity.com/paper/2109.03150/) (2021)
- [ResAct: Reinforcing Long-term Engagement in Sequential Recommendation with Residual Actor](https://mlsanity.com/paper/2206.02620/) (2022)
- [Deep Page-Level Interest Network in Reinforcement Learning for Ads Allocation](https://mlsanity.com/paper/2204.00377/) (2022)
- [Hierarchically Constrained Adaptive Ad Exposure in Feeds](https://mlsanity.com/paper/2205.15759/) (2022)
- [Hybrid Transfer in Deep Reinforcement Learning for Ads Allocation](https://mlsanity.com/paper/2204.11589/) (2022)
- [SampleLLM: Optimizing Tabular Data Synthesis in Recommendations](https://mlsanity.com/paper/2501.16125/) (2025)
- [Exploring Recommender System Evaluation: A Multi-Modal User Agent Framework for A/B Testing](https://mlsanity.com/paper/2601.04554/) (2026)
- [Learning List-wise Representation in Reinforcement Learning for Ads Allocation with Multiple Auxiliary Tasks](https://mlsanity.com/paper/2204.00888/) (2022)
- [RL-MPCA: A Reinforcement Learning Based Multi-Phase Computation Allocation Approach for Recommender Systems](https://mlsanity.com/paper/2401.01369/) (2023)
- [DARLR: Dual-Agent Offline Reinforcement Learning for Recommender Systems with Dynamic Reward](https://mlsanity.com/paper/2505.07257/) (2025)
- [Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems](https://mlsanity.com/paper/2506.23090/) (2025)
- [MDDL: A Framework for Reinforcement Learning-based Position Allocation in Multi-Channel Feed](https://mlsanity.com/paper/2304.09087/) (2023)
- [Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest](https://mlsanity.com/paper/2509.05292/) (2025)
- [C2:Cross learning module enhanced decision transformer with Constraint-aware loss for auto-bidding](https://mlsanity.com/paper/2601.20257/) (2026)
- [Fair Effect Attribution in Parallel Online Experiments](https://mlsanity.com/paper/2210.08338/) (2022)
- [Towards Validating Long-Term User Feedbacks in Interactive Recommendation Systems](https://mlsanity.com/paper/2308.11137/) (2023)
- [MaRCA: Multi-Agent Reinforcement Learning for Dynamic Computation Allocation in Large-Scale Recommender Systems](https://mlsanity.com/paper/2512.24325/) (2025)
- [One Model, Two Markets: Bid-Aware Generative Recommendation](https://mlsanity.com/paper/2603.22231/) (2026)
- [Constrained user-item allocation for e-commerce marketing campaigns](https://mlsanity.com/paper/2606.09623/) (2026)

## Abstract (arXiv)

Online recommendation and advertising are two major income channels for online recommendation platforms (e.g. e-commerce and news feed site). However, most platforms optimize recommending and advertising strategies by different teams separately via different techniques, which may lead to suboptimal overall performances. To this end, in this paper, we propose a novel two-level reinforcement learning framework to jointly optimize the recommending and advertising strategies, where the first level generates a list of recommendations to optimize user experience in the long run; then the second level inserts ads into the recommendation list that can balance the immediate advertising revenue from advertisers and the negative influence of ads on long-term user experience. To be specific, the first level tackles high combinatorial action space problem that selects a subset items from the large item space; while the second level determines three internally related tasks, i.e., (i) whether to insert an ad, and if yes, (ii) the optimal ad and (iii) the optimal location to insert. The experimental results based on real-world data demonstrate the effectiveness of the proposed framework. We have released the implementation code to ease reproductivity.