mlsanityData to 14 Oct 2026

Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation

arXiv 1911.03845 ↗First posted 10 Nov 2019cs.LG cs.IR stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning is well suited for optimizing policies of recommender systems. Current solutions mostly focus on model-free approaches, which require frequent interactions with the real environment, and thus are expensive in model learning. Offline evaluation methods, such as importance sampling, can alleviate such limitations, but usually request a large amount of logged data and do not work well when the action space is large. In this work, we propose a model-based reinforcement learning solution which models user-agent interaction for offline policy learning via a generative adversarial network. To reduce bias in the learned model and policy, we use a discriminator to evaluate the quality of generated data and scale the generated rewards. Our theoretical analysis and empirical evaluations demonstrate the effectiveness of our solution in learning policies from the offline and generated data.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 4; 2023: 14; 2024: 5; 2025: 1; 2026: 1.

33 papers.

Sort:

By area:

  1. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  2. 2023Optimizing DDPM Sampling with Shortcut Fine-Tuningcited by 81
  3. 2021Mismatched No More: Joint Model-Policy Optimization for Model-Based RLcited by 36
  4. 2022Estimating and Penalizing Induced Preference Shifts in Recommender Systemscited by 29
  5. 2023Multi-Task Recommendations with Reinforcement Learningcited by 26
  6. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 21
  7. 2023A General Offline Reinforcement Learning Framework for Interactive Recommendationcited by 18
  8. 2023Generative Slate Recommendation with Reinforcement Learningcited by 13
  9. 2023Causal Decision Transformer for Recommender Systems via Offline Reinforcement Learningcited by 11
  10. 2021Reinforcement Learning to Optimize Lifetime Value in Cold-Start Recommendationcited by 9
  11. 2023Beyond Black-Box Advice: Learning-Augmented Algorithms for MDPs with Q-Value Predictionscited by 8
  12. 2021Generative Adversarial Reward Learning for Generalized Behavior Tendency Inferencecited by 6
21 more
  1. 2022Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Controlcited by 6
  2. 2023ReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learningcited by 6
  3. 2021RL4RS: A Real-World Dataset for Reinforcement Learning based Recommender Systemcited by 5
  4. 2022Synthetic Data-Based Simulators for Recommender Systems: A Surveycited by 5
  5. 2021Reinforcement Learning with a Disentangled Universal Value Function for Item Recommendationcited by 4
  6. 2024Navigating Data Corruption in Machine Learning: Balancing Quality, Quantity, and Imputation Strategiescited by 4
  7. 2023Model-based Offline Policy Optimization with Adversarial Networkcited by 3
  8. 2024Retentive Decision Transformer with Adaptive Masking for Reinforcement Learning based Recommendation Systemscited by 3
  9. 2023Effective Dimension in Bandit Problems under Censorshipcited by 2
  10. 2024On Minimizing Adversarial Counterfactual Error in Adversarial RLcited by 2
  11. 2021Conditional Sequential Slate Optimizationcited by 1
  12. 2023Model-free Reinforcement Learning with Stochastic Reward Stabilization for Recommender Systemscited by 1
  13. 2022A generative recommender system with GMM prior for cancer drug generation and sensitivity predictioncited by 0
  14. 2023Robust Reinforcement Learning Objectives for Sequential Recommender Systemscited by 0
  15. 2023Reward Dropout Improves Control: Bi-objective Perspective on Reinforced LMcited by 0
  16. 2023A General Neural Causal Model for Interactive Recommendationcited by 0
  17. 2023Adversarial Batch Inverse Reinforcement Learning: Learn to Reward from Imperfect Demonstration for Interactive Recommendationcited by 0
  18. 2024Aligning GPTRec with Beyond-Accuracy Goals with Reinforcement Learningcited by 0
  19. 2024Federated Control in Markov Decision Processescited by 0
  20. 2025AdvKT: An Adversarial Multi-Step Training Framework for Knowledge Tracingcited by 0
  21. 2026Risk-Averse Wasserstein Distributionally Robust Online Learningcited by 0