# Hybrid Transfer in Deep Reinforcement Learning for Ads Allocation

arXiv: https://arxiv.org/abs/2204.11589 · HTML: https://mlsanity.com/paper/2204.11589/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 2 Apr 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distributional Reinforcement Learning with Quantile Regression](https://mlsanity.com/paper/1710.10044/) (2017)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systems](https://mlsanity.com/paper/1909.03602/) (2019)
- [Jointly Learning to Recommend and Advertise](https://mlsanity.com/paper/2003.00097/) (2020)
- [Learning to Collaborate: Multi-Scenario Ranking via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1809.06260/) (2018)
- [Cross DQN: Cross Deep Q Network for Ads Allocation in Feed](https://mlsanity.com/paper/2109.04353/) (2021)
- [Importance Weighted Transfer of Samples in Reinforcement Learning](https://mlsanity.com/paper/1805.10886/) (2018)
- [REPAINT: Knowledge Transfer in Deep Reinforcement Learning](https://mlsanity.com/paper/2011.11827/) (2020)
- [Exploration in Online Advertising Systems with Deep Uncertainty-Aware Learning](https://mlsanity.com/paper/2012.02298/) (2020)

## Built on by (most cited first, 5 of 5)

- [Deep Automated Mechanism Design for Integrating Ad Auction and Allocation in Feed](https://mlsanity.com/paper/2401.01656/) (2024)
- [Ads Supply Personalization via Doubly Robust Learning](https://mlsanity.com/paper/2410.12799/) (2024)
- [Session-Level Dynamic Ad Load Optimization using Offline Robust Reinforcement Learning](https://mlsanity.com/paper/2501.05591/) (2025)
- [MDDL: A Framework for Reinforcement Learning-based Position Allocation in Multi-Channel Feed](https://mlsanity.com/paper/2304.09087/) (2023)
- [Uniboost: Global Coordination with Value Alignment for Fair and Efficient Traffic Allocation](https://mlsanity.com/paper/2605.26424/) (2026)

## Abstract (arXiv)

Ads allocation, which involves allocating ads and organic items to limited slots in feed with the purpose of maximizing platform revenue, has become a research hotspot. Notice that, e-commerce platforms usually have multiple entrances for different categories and some entrances have few visits. Data from these entrances has low coverage, which makes it difficult for the agent to learn. To address this challenge, we propose Similarity-based Hybrid Transfer for Ads Allocation (SHTAA), which effectively transfers samples as well as knowledge from data-rich entrance to data-poor entrance. Specifically, we define an uncertainty-aware similarity for MDP to estimate the similarity of MDP for different entrances. Based on this similarity, we design a hybrid transfer method, including instance transfer and strategy transfer, to efficiently transfer samples and knowledge from one entrance to another. Both offline and online experiments on Meituan food delivery platform demonstrate that the proposed method could achieve better performance for data-poor entrance and increase the revenue for the platform.