# Retentive Relevance: Capturing Long-Term User Value in Recommendation Systems

arXiv: https://arxiv.org/abs/2510.07621 · HTML: https://mlsanity.com/paper/2510.07621/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 60 references that are in the map.
- First posted on arXiv 8 Oct 2025.

## Builds on (most cited first, 12 of 12)

- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Recommendations as Treatments: Debiasing Learning and Evaluation](https://mlsanity.com/paper/1602.05352/) (2016)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [Unbiased Learning-to-Rank with Biased Feedback](https://mlsanity.com/paper/1608.04468/) (2016)
- [DKN: Deep Knowledge-Aware Network for News Recommendation](https://mlsanity.com/paper/1801.08284/) (2018)
- [MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation](https://mlsanity.com/paper/1908.00413/) (2019)
- [Reinforcing User Retention in a Billion Scale Short Video Recommender System](https://mlsanity.com/paper/2302.01724/) (2023)
- [A Comprehensive Review of Recommender Systems: Transitioning from Theory to Practice](https://mlsanity.com/paper/2407.13699/) (2024)
- [Do Social Explanations Work? Studying and Modeling the Effects of Social Explanations in Recommender Systems](https://mlsanity.com/paper/1304.3405/) (2013)
- [Modeling User Retention through Generative Flow Networks](https://mlsanity.com/paper/2406.06043/) (2024)
- [AURO: Reinforcement Learning for Adaptive User Retention Optimization in Recommender Systems](https://mlsanity.com/paper/2310.03984/) (2023)
- [A Survey on Deep Neural Networks in Collaborative Filtering Recommendation Systems](https://mlsanity.com/paper/2412.01378/) (2024)

## Built on by (most cited first, 1 of 1)

- [DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search](https://mlsanity.com/paper/2608.25635/) (2026)

## Abstract (arXiv)

Recommendation systems have traditionally relied on short-term engagement signals, such as clicks and likes, to personalize content. However, these signals are often noisy, sparse, and insufficient for capturing long-term user satisfaction and retention. We introduce Retentive Relevance, a novel content-level survey-based feedback measure that directly assesses users' intent to return to the platform for similar content. Unlike other survey measures that focus on immediate satisfaction, Retentive Relevance targets forward-looking behavioral intentions, capturing longer term user intentions and providing a stronger predictor of retention. We validate Retentive Relevance using psychometric methods, establishing its convergent, discriminant, and behavioral validity. Through large-scale offline modeling, we show that Retentive Relevance significantly outperforms both engagement signals and other survey measures in predicting next-day retention, especially for users with limited historical engagement. We develop a production-ready proxy model that integrates Retentive Relevance into the final stage of a multi-stage ranking system on a social media platform. Calibrated score adjustments based on this model yield substantial improvements in engagement, and retention, while reducing exposure to low-quality content, as demonstrated by large-scale A/B experiments. This work provides the first empirically validated framework linking content-level user perceptions to retention outcomes in production systems. We offer a scalable, user-centered solution that advances both platform growth and user experience. Our work has broad implications for responsible AI development.