# Deep Reinforcement Learning for List-wise Recommendations

arXiv: https://arxiv.org/abs/1801.00209 · HTML: https://mlsanity.com/paper/1801.00209/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Dec 2017.

## Built on by (most cited first, 30 of 35)

- [Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learning](https://mlsanity.com/paper/2505.14069/) (2025)
- [Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction](https://mlsanity.com/paper/2109.01165/) (2021)
- [Multi-Task Recommendations with Reinforcement Learning](https://mlsanity.com/paper/2302.03328/) (2023)
- [Top-N Recommendation with Counterfactual User Preference Simulation](https://mlsanity.com/paper/2109.02444/) (2021)
- [A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions](https://mlsanity.com/paper/2109.03540/) (2021)
- [Measuring Recommender System Effects with Simulated Users](https://mlsanity.com/paper/2101.04526/) (2021)
- [Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation](https://mlsanity.com/paper/2109.08331/) (2021)
- [Two-Stage Constrained Actor-Critic for Short Video Recommendation](https://mlsanity.com/paper/2302.01680/) (2023)
- [Reinforcing User Retention in a Billion Scale Short Video Recommender System](https://mlsanity.com/paper/2302.01724/) (2023)
- [Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignment](https://mlsanity.com/paper/2106.00517/) (2021)
- [Reinforcement Learning to Optimize Lifetime Value in Cold-Start Recommendation](https://mlsanity.com/paper/2108.09141/) (2021)
- [Sequential/Session-based Recommendations: Challenges, Approaches, Applications and Opportunities](https://mlsanity.com/paper/2205.10759/) (2022)
- [Deep Reinforcement Learning for Autonomous Cyber Defence: A Survey](https://mlsanity.com/paper/2310.07745/) (2023)
- [RL4RS: A Real-World Dataset for Reinforcement Learning based Recommender System](https://mlsanity.com/paper/2110.11073/) (2021)
- [Reinforcement Learning with a Disentangled Universal Value Function for Item Recommendation](https://mlsanity.com/paper/2104.02981/) (2021)
- [RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm](https://mlsanity.com/paper/2301.02807/) (2023)
- [GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks](https://mlsanity.com/paper/2506.16114/) (2025)
- [RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systems](https://mlsanity.com/paper/2508.00201/) (2025)
- [Time-Constrained Recommendations: Reinforcement Learning Strategies for E-Commerce](https://mlsanity.com/paper/2512.13726/) (2025)
- [Constrained Reinforcement Learning for Short Video Recommendation](https://mlsanity.com/paper/2205.13248/) (2022)
- [Cooperative Multi-Agent Deep Reinforcement Learning in Content Ranking Optimization](https://mlsanity.com/paper/2408.04251/) (2024)
- [Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems](https://mlsanity.com/paper/2506.23090/) (2025)
- [FINN.no Slates Dataset: A new Sequential Dataset Logging Interactions, allViewed Items and Click Responses/No-Click for Recommender Systems Research](https://mlsanity.com/paper/2111.03340/) (2021)
- [Sequential Search with Off-Policy Reinforcement Learning](https://mlsanity.com/paper/2202.00245/) (2022)
- [AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation](https://mlsanity.com/paper/2308.06965/) (2023)
- [Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation](https://mlsanity.com/paper/2409.07416/) (2024)
- [Causally-Informed Reinforcement Learning for Adaptive Emotion-Aware Social Media Recommendation](https://mlsanity.com/paper/2511.14768/) (2025)
- [SAGE: Sequence-level Adaptive Gradient Evolution for Generative Recommendation](https://mlsanity.com/paper/2601.21452/) (2026)
- [Deep Reinforcement Learning-Based Product Recommender for Online Advertising](https://mlsanity.com/paper/2102.00333/) (2021)
- [D2RLIR : an improved and diversified ranking function in interactive recommendation systems based on deep reinforcement learning](https://mlsanity.com/paper/2110.15089/) (2021)

## Abstract (arXiv)

Recommender systems play a crucial role in mitigating the problem of information overload by suggesting users' personalized items or services. The vast majority of traditional recommender systems consider the recommendation procedure as a static process and make recommendations following a fixed strategy. In this paper, we propose a novel recommender system with the capability of continuously improving its strategies during the interactions with users. We model the sequential interactions between users and a recommender system as a Markov Decision Process (MDP) and leverage Reinforcement Learning (RL) to automatically learn the optimal strategies via recommending trial-and-error items and receiving reinforcements of these items from users' feedbacks. In particular, we introduce an online user-agent interacting environment simulator, which can pre-train and evaluate model parameters offline before applying the model online. Moreover, we validate the importance of list-wise recommendations during the interactions between users and agent, and develop a novel approach to incorporate them into the proposed framework LIRD for list-wide recommendations. The experimental results based on a real-world e-commerce dataset demonstrate the effectiveness of the proposed framework.