# Deep Reinforcement Learning based Recommendation with Explicit User-Item Interactions Modeling

arXiv: https://arxiv.org/abs/1810.12027 · HTML: https://mlsanity.com/paper/1810.12027/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2018.

## Built on by (most cited first, 30 of 31)

- [AndroidEnv: A Reinforcement Learning Platform for Android](https://mlsanity.com/paper/2105.13231/) (2021)
- [Multi-Task Recommendations with Reinforcement Learning](https://mlsanity.com/paper/2302.03328/) (2023)
- [User Tampering in Reinforcement Learning Recommender Systems](https://mlsanity.com/paper/2109.04083/) (2021)
- [A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions](https://mlsanity.com/paper/2109.03540/) (2021)
- [Reward Shaping for User Satisfaction in a REINFORCE Recommender](https://mlsanity.com/paper/2209.15166/) (2022)
- [Sim2Rec: A Simulator-based Decision-making Approach to Optimize Real-World Long-term User Engagement in Sequential Recommender Systems](https://mlsanity.com/paper/2305.04832/) (2023)
- [Marketing Budget Allocation with Offline Constrained Deep Reinforcement Learning](https://mlsanity.com/paper/2309.02669/) (2023)
- [RePlay: a Recommendation Framework for Experimentation and Production Use](https://mlsanity.com/paper/2409.07272/) (2024)
- [Time-Constrained Recommendations: Reinforcement Learning Strategies for E-Commerce](https://mlsanity.com/paper/2512.13726/) (2025)
- [Off Environment Evaluation Using Convex Risk Minimization](https://mlsanity.com/paper/2112.11532/) (2021)
- [Sequential Condition Evolved Interaction Knowledge Graph for Traditional Chinese Medicine Recommendation](https://mlsanity.com/paper/2305.17866/) (2023)
- [UOEP: User-Oriented Exploration Policy for Enhancing Long-Term User Experiences in Recommender Systems](https://mlsanity.com/paper/2401.09034/) (2024)
- [Towards Optimal Offline Reinforcement Learning](https://mlsanity.com/paper/2503.12283/) (2025)
- [Click-Through Rate Prediction Using Graph Neural Networks and Online Learning](https://mlsanity.com/paper/2105.03811/) (2021)
- [AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation](https://mlsanity.com/paper/2308.06965/) (2023)
- [Model-free Reinforcement Learning with Stochastic Reward Stabilization for Recommender Systems](https://mlsanity.com/paper/2308.13246/) (2023)
- [Breaking the Filter Bubble: A Semantic Pareto-DQN Framework for Multi-Objective Recommendation](https://mlsanity.com/paper/2606.24042/) (2026)
- [Deep Reinforcement Learning-Based Product Recommender for Online Advertising](https://mlsanity.com/paper/2102.00333/) (2021)
- [Contextual Bandits with Sparse Data in Web setting](https://mlsanity.com/paper/2105.02873/) (2021)
- [SINGA-Easy: An Easy-to-Use Framework for MultiModal Analysis](https://mlsanity.com/paper/2108.02572/) (2021)
- [VisualTextRank: Unsupervised Graph-based Content Extraction for Automating Ad Text to Image Search](https://mlsanity.com/paper/2108.02725/) (2021)
- [Text is NOT Enough: Integrating Visual Impressions into Open-domain Dialogue Generation](https://mlsanity.com/paper/2109.05778/) (2021)
- [D2RLIR : an improved and diversified ranking function in interactive recommendation systems based on deep reinforcement learning](https://mlsanity.com/paper/2110.15089/) (2021)
- [Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors](https://mlsanity.com/paper/2202.02830/) (2022)
- [Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks](https://mlsanity.com/paper/2203.11011/) (2022)
- [Explanation-Guided Fairness Testing through Genetic Algorithm](https://mlsanity.com/paper/2205.08335/) (2022)
- [Using Graph Neural Networks for Program Termination](https://mlsanity.com/paper/2207.14648/) (2022)
- [Adaptive Interventions for Global Health: A Case Study of Malaria](https://mlsanity.com/paper/2303.02075/) (2023)
- [Tribe or Not? Critical Inspection of Group Differences Using TribalGram](https://mlsanity.com/paper/2303.09664/) (2023)
- [Towards Validating Long-Term User Feedbacks in Interactive Recommendation Systems](https://mlsanity.com/paper/2308.11137/) (2023)

## Abstract (arXiv)

Recommendation is crucial in both academia and industry, and various techniques are proposed such as content-based collaborative filtering, matrix factorization, logistic regression, factorization machines, neural networks and multi-armed bandits. However, most of the previous studies suffer from two limitations: (1) considering the recommendation as a static procedure and ignoring the dynamic interactive nature between users and the recommender systems, (2) focusing on the immediate feedback of recommended items and neglecting the long-term rewards. To address the two limitations, in this paper we propose a novel recommendation framework based on deep reinforcement learning, called DRR. The DRR framework treats recommendation as a sequential decision making procedure and adopts an "Actor-Critic" reinforcement learning scheme to model the interactions between the users and recommender systems, which can consider both the dynamic adaptation and long-term rewards. Furthermore, a state representation module is incorporated into DRR, which can explicitly capture the interactions between items and users. Three instantiation structures are developed. Extensive experiments on four real-world datasets are conducted under both the offline and online evaluation settings. The experimental results demonstrate the proposed DRR method indeed outperforms the state-of-the-art competitors.