# Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning

arXiv: https://arxiv.org/abs/2607.14192 · HTML: https://mlsanity.com/paper/2607.14192/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 60 references that are in the map.
- First posted on arXiv 15 Jul 2026.

## Builds on (most cited first, 30 of 32)

- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [Controllable Multi-Interest Framework for Recommendation](https://mlsanity.com/paper/2005.09347/) (2020)
- [Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems](https://mlsanity.com/paper/1902.05570/) (2019)
- [RankMixer: Scaling Up Ranking Models in Industrial Recommenders](https://mlsanity.com/paper/2507.15551/) (2025)
- [TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou](https://mlsanity.com/paper/2302.02352/) (2023)
- [TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest](https://mlsanity.com/paper/2306.00248/) (2023)
- [M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework](https://mlsanity.com/paper/2404.18465/) (2024)
- [TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou](https://mlsanity.com/paper/2407.16357/) (2024)
- [Multi-Task Recommendations with Reinforcement Learning](https://mlsanity.com/paper/2302.03328/) (2023)
- [Multi-Task Fusion via Reinforcement Learning for Long-Term User Satisfaction in Recommender Systems](https://mlsanity.com/paper/2208.04560/) (2022)
- [Real-time Short Video Recommendation on Mobile Devices](https://mlsanity.com/paper/2208.09577/) (2022)
- [PrefRec: Recommender Systems with Human Preferences for Reinforcing Long-term User Engagement](https://mlsanity.com/paper/2212.02779/) (2022)
- [Re4: Learning to Re-contrast, Re-attend, Re-construct for Multi-interest Recommendation](https://mlsanity.com/paper/2208.08011/) (2022)
- [Generative Recommendation with Semantic IDs: A Practitioner's Handbook](https://mlsanity.com/paper/2507.22224/) (2025)
- [Reinforcing User Retention in a Billion Scale Short Video Recommender System](https://mlsanity.com/paper/2302.01724/) (2023)
- [User Retention-oriented Recommendation with Decision Transformer](https://mlsanity.com/paper/2303.06347/) (2023)
- [OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search](https://mlsanity.com/paper/2404.16260/) (2024)
- [When Multi-Level Meets Multi-Interest: A Multi-Grained Neural Model for Sequential Recommendation](https://mlsanity.com/paper/2205.01286/) (2022)
- [On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems](https://mlsanity.com/paper/2308.11336/) (2023)
- [Sequential Recommendation for Optimizing Both Immediate Feedback and Long-term Retention](https://mlsanity.com/paper/2404.03637/) (2024)
- [Scaling Transformers for Discriminative Recommendation via Generative Pretraining](https://mlsanity.com/paper/2506.03699/) (2025)
- [Maximizing Cumulative User Engagement in Sequential Recommendation: An Online Optimization Perspective](https://mlsanity.com/paper/2006.04520/) (2020)
- [Optimizing Long-term Value for Auction-Based Recommender Systems via On-Policy Reinforcement Learning](https://mlsanity.com/paper/2305.13747/) (2023)
- [Deep Task-specific Bottom Representation Network for Multi-Task Recommendation](https://mlsanity.com/paper/2308.05996/) (2023)
- [AURO: Reinforcement Learning for Adaptive User Retention Optimization in Recommender Systems](https://mlsanity.com/paper/2310.03984/) (2023)

## Built on by (most cited first, 1 of 1)

- [DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search](https://mlsanity.com/paper/2608.25635/) (2026)

## Abstract (arXiv)

As recommender systems mature in the past few years, their optimization objectives have evolved from a primary focusing on short-term behavioral signals to a broader emphasis on long-term user engagement and retention. However, directly optimizing retention is difficult because return signals are sparse, delayed, and only partially attributable to earlier recommendations. Prior work has addressed this challenge with sequential modeling and reinforcement learning, but these approaches typically require task specific reward engineering, substantial computational overhead, and surface specific implementations that are difficult to generalize. In this paper, we present a unified, model-agnostic downstream reward framework for optimizing long-term user value in large-scale recommendation systems. First, we formulate the downstream reward learning problem and develop an offline screening framework to identify session level behaviors that are both observable early and predictive of future retention. We then propose several model-agnostic downstream rewards signals derived from observed user action patterns across multiple sources. We further discuss the engineering effort to productionize the proposed rewards derivations and challenges we faced when adding them to our ranking models. Online A/B experiments demonstrate consistent improvements in engagement and retention-related metrics, and the framework has been deployed across multiple Pinterest surfaces, including Homefeed, Related Pins, Search, and Notifications.