mlsanityData to 7 Oct 2026

A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions

arXiv 2109.03540 ↗First posted 8 Sep 2021cs.IR cs.AIcited by 19

Not on the map. Why?

Abstract (arXiv)

In light of the emergence of deep reinforcement learning (DRL) in recommender systems research and several fruitful results in recent years, this survey aims to provide a timely and comprehensive overview of the recent trends of deep reinforcement learning in recommender systems. We start with the motivation of applying DRL in recommender systems. Then, we provide a taxonomy of current DRL-based recommender systems and a summary of existing methods. We discuss emerging topics and open issues, and provide our perspective on advancing the domain. This survey serves as introductory material for readers from academia and industry into the topic and identifies notable opportunities for further research.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 73 of its 132 references that are in the map. First posted on arXiv 8 Sep 2021.

Builds on

73 of 132 references.

References · 73

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  8. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  9. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  10. 2016Generative Adversarial Imitation Learningcited by 1,576
  11. 2015Prioritized Experience Replaycited by 1,070
  12. 2017Constrained Policy Optimizationcited by 928
38 more
  1. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  2. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 473
  3. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  4. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  5. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  6. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  7. 2016Learning to reinforcement learncited by 369
  8. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  9. 2016Recommendations as Treatments: Debiasing Learning and Evaluationcited by 246
  10. 2018Selective Experience Replay for Lifelong Learningcited by 239
  11. 2020Bias and Debias in Recommender System: A Survey and Future Directionscited by 239
  12. 2018Explainable Recommendation: A Survey and New Perspectivescited by 210
  13. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  14. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  15. 2021Reinforcement learning based recommender systems: A surveycited by 187
  16. 2018Top-K Off-Policy Correction for a REINFORCE Recommender Systemcited by 181
  17. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  18. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
  19. 2018Graph Convolutional Reinforcement Learningcited by 148
  20. 2019Causal Discovery with Reinforcement Learningcited by 145
  21. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 121
  22. 2019Knowledge Graph Convolutional Networks for Recommender Systemscited by 121
  23. 2019Reinforcement Knowledge Graph Reasoning for Explainable Recommendationcited by 102
  24. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
  25. 2018Deep Reinforcement Learning for Page-wise Recommendationscited by 99
  26. 2019RecSim: A Configurable Simulation Platform for Recommender Systemscited by 91
  27. 2017Action Branching Architectures for Deep Reinforcement Learningcited by 90
  28. 2019Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systemscited by 90
  29. 2020RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Realcited by 89
  30. 2018Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learningcited by 80
  31. 2017Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigationcited by 74
  32. 2018Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learningcited by 71
  33. 2018RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertisingcited by 65
  34. 2018Generative Adversarial User Model for Reinforcement Learning Based Recommendation Systemcited by 64
  35. 2019Causal Reasoning from Meta-reinforcement Learningcited by 53
  36. 2020Self-Supervised Reinforcement Learning for Recommender Systemscited by 49
  37. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
  38. 2019DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systemscited by 44

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 5; 2024: 5; 2025: 4; 2026: 1.

19 papers.

Sort:

By area:

  1. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  2. 2023Searching for High-Value Molecules Using Reinforcement Learning and Transformerscited by 18
  3. 2023Reinforcement Learning for Generative AI: A Surveycited by 11
  4. 2022Synthetic Data-Based Simulators for Recommender Systems: A Surveycited by 5
  5. 2021Pessimistic Model Selection for Offline Deep Reinforcement Learningcited by 3
  6. 2023Marketing Budget Allocation with Offline Constrained Deep Reinforcement Learningcited by 3
  7. 2025RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systemscited by 3
  8. 2025Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systemscited by 2
  9. 2023Bi-directional personalization reinforcement learning-based architecture with active learning using a multi-model data service for the travel nursing industrycited by 1
  10. 2023Model-free Reinforcement Learning with Stochastic Reward Stabilization for Recommender Systemscited by 1
  11. 2024FedSlate:A Federated Deep Reinforcement Learning Recommender Systemcited by 1
  12. 2024Toward Debugging Deep Reinforcement Learning Programs with RLExplorercited by 1
7 more
  1. 2022Fine-Grained Session Recommendations in E-commerce using Deep Reinforcement Learningcited by 0
  2. 2022Situating Recommender Systems in Practice: Towards Inductive Learning and Incremental Updatescited by 0
  3. 2024Position Paper: Why the Shooting in the Dark Method Dominates Recommender Systems Practice; A Call to Abandon Anti-Utopian Thinkingcited by 0
  4. 2024Debiased Model-based Interactive Recommendationcited by 0
  5. 2024An Advantage-based Optimization Method for Reinforcement Learning in Large Action Spacecited by 0
  6. 2025Statistical Guarantees for Offline Domain Randomizationcited by 0
  7. 2026Developing AI Agents with Simulated Data: Why, what, and how?cited by 0