mlsanityData to 7 Oct 2026

Generative Adversarial User Model for Reinforcement Learning Based Recommendation System

arXiv 1812.10613 ↗First posted 27 Dec 2018cs.LG cs.IR stat.MLcited by 64

Not on the map. Why?

Abstract (arXiv)

There are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems. In this setting, an online user is the environment; neither the reward function nor the environment dynamics are clearly defined, making the application of RL challenging. In this paper, we propose a novel model-based reinforcement learning framework for recommendation systems, where we develop a generative adversarial network to imitate user behavior dynamics and learn her reward function. Using this user model as the simulation environment, we develop a novel Cascading DQN algorithm to obtain a combinatorial recommendation policy which can handle a large number of candidate items efficiently. In our experiments with real data, we show this generative adversarial user model can better explain user behavior than alternatives, and the RL policy based on this model can lead to a better long-term reward for the user and higher click rate for the system.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 11; 2023: 23; 2024: 12; 2025: 3; 2026: 3.

64 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
  3. 2023User Behavior Simulation with Large Language Model based Agentscited by 136
  4. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  5. 2023On Generative Agents in Recommendationcited by 88
  6. 2023Elastic Decision Transformercited by 66
  7. 2024A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)cited by 58
  8. 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agentcited by 46
  9. 2024On Targeted Manipulation and Deception when Optimizing LLMs for User Feedbackcited by 45
  10. 2024Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectivescited by 30
  11. 2022Estimating and Penalizing Induced Preference Shifts in Recommender Systemscited by 29
  12. 2024Prototypical Reward Network for Data-Efficient RLHFcited by 22
38 more
  1. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 19
  2. 2023A General Offline Reinforcement Learning Framework for Interactive Recommendationcited by 18
  3. 2022Rethinking ValueDice: Does It Really Improve Performance?cited by 15
  4. 2021Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learningcited by 14
  5. 2022Adversarial Counterfactual Environment Model Learningcited by 12
  6. 2023Generative Slate Recommendation with Reinforcement Learningcited by 12
  7. 2024AI Agent for Education: von Neumann Multi-Agent System Frameworkcited by 12
  8. 2024SUBER: An RL Environment with Simulated Human Behavior for Recommender Systemscited by 11
  9. 2023A Versatile Multi-Agent Reinforcement Learning Benchmark for Inventory Managementcited by 9
  10. 2021Supervised Advantage Actor-Critic for Recommender Systemscited by 8
  11. 2023Optimizing Audio Recommendations for the Long-Term: A Reinforcement Learning Perspectivecited by 8
  12. 2023How To Guide Your Learner: Imitation Learning with Active Adaptive Expert Involvementcited by 8
  13. 2021Generative Adversarial Reward Learning for Generalized Behavior Tendency Inferencecited by 6
  14. 2022Meta Policy Learning for Cold-Start Conversational Recommendationcited by 6
  15. 2023Optimizing Long-term Value for Auction-Based Recommender Systems via On-Policy Reinforcement Learningcited by 6
  16. 2024Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximationcited by 6
  17. 2021RL4RS: A Real-World Dataset for Reinforcement Learning based Recommender Systemcited by 5
  18. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 5
  19. 2022Synthetic Data-Based Simulators for Recommender Systems: A Surveycited by 5
  20. 2023Sim2Rec: A Simulator-based Decision-making Approach to Optimize Real-World Long-term User Engagement in Sequential Recommender Systemscited by 5
  21. 2023Offline Imitation from Observation via Primal Wasserstein State Occupancy Matchingcited by 5
  22. 2021Reinforcement Learning with a Disentangled Universal Value Function for Item Recommendationcited by 4
  23. 2022CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrationscited by 4
  24. 2023Multi-Resolution Diffusion for Privacy-Sensitive Recommender Systemscited by 4
  25. 2025Beyond Static Testbeds: An Interaction-Centric Agent Simulation Platform for Dynamic Recommender Systemscited by 4
  26. 2021Inferring Substitutable and Complementary Products with Knowledge-Aware Path Reasoning based on Dynamic Policy Networkcited by 3
  27. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectoriescited by 3
  28. 2023Efficient Reinforcement Learning via Decoupling Exploration and Utilizationcited by 3
  29. 2024Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functionscited by 3
  30. 2025Inducing Personality in LLM-Based Honeypot Agents: Measuring the Effect on Human-Like Agenda Generationcited by 3
  31. 2026Exploring Recommender System Evaluation: A Multi-Modal User Agent Framework for A/B Testingcited by 3
  32. 2023Backstepping Temporal Difference Learningcited by 2
  33. 2023Dual Policy Learning for Aggregation Optimization in Graph Neural Network-based Recommender Systemscited by 2
  34. 2024Online Poisoning Attack Against Reinforcement Learning under Black-box Environmentscited by 2
  35. 2022A Scalable Recommendation Engine for New Users and Itemscited by 1
  36. 2023Simulating News Recommendation Ecosystem for Fun and Profitcited by 1
  37. 2023Hierarchical Reinforcement Learning for Modeling User Novelty-Seeking Intent in Recommender Systemscited by 1
  38. 2023Model-free Reinforcement Learning with Stochastic Reward Stabilization for Recommender Systemscited by 1

50 most cited of 64 All 64 (JSON)