mlsanityData to 14 Oct 2026

RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systems

arXiv 2508.00201 ↗First posted 31 Jul 2025cs.LGcited by 3

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Existing web-scale recommendation systems commonly use supervised learning methods that prioritize immediate user feedback. Although reinforcement learning (RL) offers a solution to optimize longer-term goals, such as in-session engagement, applying it at web scale is challenging due to the extremely large action space and engineering complexity. In this paper, we introduce RecoMind, a simulator-based RL framework designed for the effective optimization of session-based goals at web-scale. RecoMind leverages existing recommendation models to establish a simulation environment and to bootstrap the RL policy to optimize immediate user interactions from the outset. This method integrates well with existing industry pipelines, simplifying the training and deployment of RL policies. Additionally, RecoMind introduces a custom exploration strategy to efficiently explore web-scale action spaces with hundreds of millions of items. We evaluated RecoMind through extensive offline simulations and online A/B testing on a video streaming platform. Both methods showed that the RL policy trained using RecoMind significantly outperforms traditional supervised learning recommendation approaches in in-session user satisfaction. In online A/B tests, the RL policy increased videos watched for more than 10 seconds by 15.81% and improved session depth by 4.71% for sessions with at least 10 interactions. As a result, RecoMind presents a systematic and scalable approach for embedding RL into web-scale recommendation systems, showing great promise for optimizing session-based user satisfaction.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 47 references that are in the map. First posted on arXiv 31 Jul 2025.

Builds on

22 of 47 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2015Prioritized Experience Replaycited by 1,099
  5. 2016Wide & Deep Learning for Recommender Systemscited by 747
  6. 2015Session-based Recommendations with Recurrent Neural Networkscited by 695
  7. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  8. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  9. 2017Neural Attentive Session-based Recommendationcited by 237
  10. 2018Top-K Off-Policy Correction for a REINFORCE Recommender Systemcited by 182
  11. 2021A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directionscited by 144
  12. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
10 more
  1. 2019RecSim: A Configurable Simulation Platform for Recommender Systemscited by 94
  2. 2019Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systemscited by 90
  3. 2017Deep Reinforcement Learning for List-wise Recommendationscited by 35
  4. 2023TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterestcited by 35
  5. 2018Deep reinforcement learning for search, recommendation, and online advertising: a surveycited by 23
  6. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 21
  7. 2022Rethinking Personalized Ranking at Pinterest: An End-to-End Approachcited by 16
  8. 2023Generative Slate Recommendation with Reinforcement Learningcited by 13
  9. 2016Partially Observable Markov Decision Process for Recommender Systemscited by 11
  10. 2019Off-policy Learning for Multiple Loggerscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Self-EvolveRec: Self-Evolving Recommender Systems with LLM-based Directional Feedbackcited by 10
  2. 2025Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterestcited by 1
  3. 2026A Production-Ready RL Framework for Personalized Utility Tuning with Pareto Sweeping in Pinterest Recommender Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM-Enhanced Reinforcement Learning for Long-Term User Satisfaction in Interactive Recommendation
  2. 2024UOEP: User-Oriented Exploration Policy for Enhancing Long-Term User Experiences in Recommender Systems
  3. 2026RecRM-Bench: Benchmarking Multidimensional Reward Modeling for Agentic Recommender Systems
  4. 2022PrefRec: Recommender Systems with Human Preferences for Reinforcing Long-term User Engagement
  5. 2024EasyRL4Rec: An Easy-to-use Library for Reinforcement Learning Based Recommender Systems
  6. 2023On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems
  7. 2025Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learning
  8. 2020Offline Meta-level Model-based Reinforcement Learning Approach for Cold-Start Recommendation