mlsanityData to 7 Oct 2026

Fine-Grained Session Recommendations in E-commerce using Deep Reinforcement Learning

arXiv 2210.15451 ↗First posted 20 Oct 2022cs.IR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Sustaining users' interest and keeping them engaged in the platform is very important for the success of an e-commerce business. A session encompasses different activities of a user between logging into the platform and logging out or making a purchase. User activities in a session can be classified into two groups: Known Intent and Unknown intent. Known intent activity pertains to the session where the intent of a user to browse/purchase a specific product can be easily captured. Whereas in unknown intent activity, the intent of the user is not known. For example, consider the scenario where a user enters the session to casually browse the products over the platform, similar to the window shopping experience in the offline setting. While recommending similar products is essential in the former, accurately understanding the intent and recommending interesting products is essential in the latter setting in order to retain a user. In this work, we focus primarily on the unknown intent setting where our objective is to recommend a sequence of products to a user in a session to sustain their interest, keep them engaged and possibly drive them towards purchase. We formulate this problem in the framework of the Markov Decision Process (MDP), a popular mathematical framework for sequential decision making and solve it using Deep Reinforcement Learning (DRL) techniques. However, training the next product recommendation is difficult in the RL paradigm due to large variance in browse/purchase behavior of the users. Therefore, we break the problem down into predicting various product attributes, where a pattern/trend can be identified and exploited to build accurate models. We show that the DRL agent provides better performance compared to a greedy strategy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 44 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

18 of 44 references.

References · 18

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  5. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  6. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 336
  7. 2017Neural Attentive Session-based Recommendationcited by 237
  8. 2021Reinforcement learning based recommender systems: A surveycited by 187
  9. 2017Recurrent Neural Networks with Top-k Gains for Session-based Recommendationscited by 165
  10. 2019Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systemscited by 90
  11. 2017Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networkscited by 84
  12. 2019FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Predictioncited by 83
6 more
  1. 2019Deep Session Interest Network for Click-Through Rate Predictioncited by 64
  2. 2020Self-Supervised Reinforcement Learning for Recommender Systemscited by 49
  3. 2017Deep Reinforcement Learning for List-wise Recommendationscited by 35
  4. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 19
  5. 2017Contextual Sequence Modeling for Recommendation with Recurrent Neural Networkscited by 11
  6. 2017Inter-Session Modeling for Session-Based Recommendationcited by 3

Built on by

Not cited yet.