# A Model-based Multi-Agent Personalized Short-Video Recommender System

arXiv: https://arxiv.org/abs/2405.01847 · HTML: https://mlsanity.com/paper/2405.01847/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 9 of 9)

- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://mlsanity.com/paper/1804.07931/) (2018)
- [Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems](https://mlsanity.com/paper/1902.05570/) (2019)
- [KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos](https://mlsanity.com/paper/2208.08696/) (2022)
- [Two-Stage Constrained Actor-Critic for Short Video Recommendation](https://mlsanity.com/paper/2302.01680/) (2023)
- [UKD: Debiasing Conversion Rate Estimation via Uncertainty-regularized Knowledge Distillation](https://mlsanity.com/paper/2201.08024/) (2022)

## Built on by (most cited first, 2 of 2)

- [Multi-Agent Video Recommenders: Evolution, Patterns, and Open Challenges](https://mlsanity.com/paper/2604.02211/) (2026)
- [VisualLens: Personalization through Task-Agnostic Visual History](https://mlsanity.com/paper/2411.16034/) (2024)

## Abstract (arXiv)

Recommender selects and presents top-K items to the user at each online request, and a recommendation session consists of several sequential requests. Formulating a recommendation session as a Markov decision process and solving it by reinforcement learning (RL) framework has attracted increasing attention from both academic and industry communities. In this paper, we propose a RL-based industrial short-video recommender ranking framework, which models and maximizes user watch-time in an environment of user multi-aspect preferences by a collaborative multi-agent formulization. Moreover, our proposed framework adopts a model-based learning approach to alleviate the sample selection bias which is a crucial but intractable problem in industrial recommender system. Extensive offline evaluations and live experiments confirm the effectiveness of our proposed method over alternatives. Our proposed approach has been deployed in our real large-scale short-video sharing platform, successfully serving over hundreds of millions users.