# A Survey on Reinforcement Learning for Recommender Systems

arXiv: https://arxiv.org/abs/2109.10665 · HTML: https://mlsanity.com/paper/2109.10665/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2021.

## Built on by (most cited first, 18 of 18)

- [A Survey on Causal Reinforcement Learning](https://mlsanity.com/paper/2302.05209/) (2023)
- [Improved Regret Bounds for Linear Adversarial MDPs via Linear Optimization](https://mlsanity.com/paper/2302.06834/) (2023)
- [SUBER: An RL Environment with Simulated Human Behavior for Recommender Systems](https://mlsanity.com/paper/2406.01631/) (2024)
- [AdaAugment: A Tuning-Free and Adaptive Approach to Enhance Data Augmentation](https://mlsanity.com/paper/2405.11467/) (2024)
- [Provable Domain Adaptation for Offline Reinforcement Learning with Limited Samples](https://mlsanity.com/paper/2408.12136/) (2024)
- [Filtering Learning Histories Enhances In-Context Reinforcement Learning](https://mlsanity.com/paper/2505.15143/) (2025)
- [Evolutionary Reinforcement Learning: A Systematic Review and Future Directions](https://mlsanity.com/paper/2402.13296/) (2024)
- [Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization](https://mlsanity.com/paper/2403.07916/) (2024)
- [A Comprehensive Survey on Deep Learning Techniques in Educational Data Mining](https://mlsanity.com/paper/2309.04761/) (2023)
- [Time-Constrained Recommendations: Reinforcement Learning Strategies for E-Commerce](https://mlsanity.com/paper/2512.13726/) (2025)
- [An Experimental Comparison Between Temporal Difference and Residual Gradient with Neural Network Approximation](https://mlsanity.com/paper/2205.12770/) (2022)
- [Model-Based Offline Reinforcement Learning with Adversarial Data Augmentation](https://mlsanity.com/paper/2503.20285/) (2025)
- [Policy-Based Trajectory Clustering in Offline Reinforcement Learning](https://mlsanity.com/paper/2506.09202/) (2025)
- [Neural Click Models for Recommender Systems](https://mlsanity.com/paper/2409.20055/) (2024)
- [Conv4Rec: A 1-by-1 Convolutional AutoEncoder for User Profiling through Joint Analysis of Implicit and Explicit Feedbacks](https://mlsanity.com/paper/2509.07499/) (2025)
- [LLM-Enhanced Reinforcement Learning for Long-Term User Satisfaction in Interactive Recommendation](https://mlsanity.com/paper/2601.19585/) (2026)
- [Distributed Zeroth-Order Policy Gradient for Networked Multi-agent Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2605.15697/) (2026)
- [Human-inspired, Task-Dimension-Guided Exploration for Efficient Learning in High Dimensions](https://mlsanity.com/paper/2609.36672/) (2026)

## Abstract (arXiv)

Recommender systems have been widely applied in different real-life scenarios to help us find useful information. In particular, Reinforcement Learning (RL) based recommender systems have become an emerging research topic in recent years, owing to the interactive nature and autonomous learning ability. Empirical results show that RL-based recommendation methods often surpass most of supervised learning methods. Nevertheless, there are various challenges of applying RL in recommender systems. To understand the challenges and relevant solutions, there should be a reference for researchers and practitioners working on RL-based recommender systems. To this end, we firstly provide a thorough overview, comparisons, and summarization of RL approaches applied in four typical recommendation scenarios, including interactive recommendation, conversational recommendatin, sequential recommendation, and explainable recommendation. Furthermore, we systematically analyze the challenges and relevant solutions on the basis of existing literature. Finally, under discussion for open issues of RL and its limitations of recommender systems, we highlight some potential research directions in this field.