mlsanityData to 7 Oct 2026

Reinforcing User Retention in a Billion Scale Short Video Recommender System

arXiv 2302.01724 ↗First posted 3 Feb 2023cs.LG cs.IRcited by 15

Not on the map. Why?

Abstract (arXiv)

Recently, short video platforms have achieved rapid user growth by recommending interesting content to users. The objective of the recommendation is to optimize user retention, thereby driving the growth of DAU (Daily Active Users). Retention is a long-term feedback after multiple interactions of users and the system, and it is hard to decompose retention reward to each item or a list of items. Thus traditional point-wise and list-wise models are not able to optimize retention. In this paper, we choose reinforcement learning methods to optimize the retention as they are designed to maximize the long-term performance. We formulate the problem as an infinite-horizon request-based Markov Decision Process, and our objective is to minimize the accumulated time interval of multiple sessions, which is equal to improving the app open frequency and user retention. However, current reinforcement learning algorithms can not be directly applied in this setting due to uncertainty, bias, and long delay time incurred by the properties of user retention. We propose a novel method, dubbed RLUR, to address the aforementioned challenges. Both offline and live experiments show that RLUR can significantly improve user retention. RLUR has been fully launched in Kuaishou app for a long time, and achieves consistent performance improvement on user retention and DAU.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 20 of its 39 references that are in the map. First posted on arXiv 3 Feb 2023.

Builds on

20 of 39 references.

References · 20

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  5. 2018Exploration by Random Network Distillationcited by 966
  6. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  7. 2021Reinforcement learning based recommender systems: A surveycited by 187
  8. 2019Reinforcement Knowledge Graph Reasoning for Explainable Recommendationcited by 102
  9. 2019Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systemscited by 90
  10. 2021Towards Long-term Fairness in Recommendationcited by 77
  11. 2018Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learningcited by 71
  12. 2022KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videoscited by 69
8 more
  1. 2022CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender Systemcited by 38
  2. 2022Deconfounding Duration Bias in Watch-time Prediction for Video Recommendationcited by 37
  3. 2017Deep Reinforcement Learning for List-wise Recommendationscited by 35
  4. 2018Large-scale Interactive Recommendation with Tree-structured Policy Gradientcited by 30
  5. 2022Multi-Task Fusion via Reinforcement Learning for Long-Term User Satisfaction in Recommender Systemscited by 22
  6. 2022Real-time Short Video Recommendation on Mobile Devicescited by 21
  7. 2022Feature-aware Diversified Re-ranking with Disentangled Representations for Relevant Recommendationcited by 11
  8. 2022ResAct: Reinforcing Long-term Engagement in Sequential Recommendation with Residual Actorcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 5; 2026: 3.

15 papers.

Sort:

By area:

  1. 2024AI Alignment with Changing and Influenceable Reward Functionscited by 33
  2. 2024EasyRL4Rec: An Easy-to-use Library for Reinforcement Learning Based Recommender Systemscited by 8
  3. 2023AURO: Reinforcement Learning for Adaptive User Retention Optimization in Recommender Systemscited by 5
  4. 2024Learned Ranking Function: From Short-term Behavior Predictions to Long-term User Satisfactioncited by 4
  5. 2024UnifiedRL: A Reinforcement Learning Algorithm Tailored for Multi-Task Fusion in Large-Scale Recommender Systemscited by 2
  6. 2025Retentive Relevance: Capturing Long-Term User Value in Recommendation Systemscited by 1
  7. 2025GRADE: Personalized Multi-Task Fusion via Group-relative Reinforcement Learning with Adaptive Dirichlet Explorationcited by 1
  8. 2026Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learningcited by 1
  9. 2024EnhancedRL: An Enhanced-State Reinforcement Learning Algorithm for Multi-Task Fusion in Recommender Systemscited by 0
  10. 2024Supervised Learning-enhanced Multi-Group Actor Critic for Live Stream Allocation in Feedcited by 0
  11. 2025MultiScale Contextual Bandits for Long Term Objectivescited by 0
  12. 2025Short-Form Video Recommendations with Multimodal Embeddings: Addressing Cold-Start and Bias Challengescited by 0
3 more
  1. 2025KLAN: Kuaishou Landing-page Adaptive Navigatorcited by 0
  2. 2026Scale over Preference: The Impact of AI-Generated Content on Online Content Ecologycited by 0
  3. 2026PPDL: A Real-world Industrial User Retention Ratio Forecasting Framework Integrating Physical Priors with Deep Learningcited by 0