# JDRec: Practical Actor-Critic Framework for Online Combinatorial Recommender System

arXiv: https://arxiv.org/abs/2207.13311 · HTML: https://mlsanity.com/paper/2207.13311/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 27 Jul 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [Fast Matrix Factorization for Online Recommendation with Implicit Feedback](https://mlsanity.com/paper/1708.05024/) (2017)
- [Learning a Deep Listwise Context Model for Ranking Refinement](https://mlsanity.com/paper/1804.05936/) (2018)
- [SetRank: Learning a Permutation-Invariant Ranking Model for Information Retrieval](https://mlsanity.com/paper/1912.05891/) (2019)
- [Learning Groupwise Multivariate Scoring Functions Using Deep Neural Networks](https://mlsanity.com/paper/1811.04415/) (2018)
- [Globally Optimized Mutual Influence Aware Ranking in E-Commerce Search](https://mlsanity.com/paper/1805.08524/) (2018)
- [Sequential Evaluation and Generation Framework for Combinatorial Recommender System](https://mlsanity.com/paper/1902.00245/) (2019)
- [Generator and Critic: A Deep Reinforcement Learning Approach for Slate Re-ranking in E-commerce](https://mlsanity.com/paper/2005.12206/) (2020)

## Built on by (most cited first, 1 of 1)

- [FlashEvaluator: Expanding Search Space with Parallel Sequence-Level Evaluation](https://mlsanity.com/paper/2603.02565/) (2026)

## Abstract (arXiv)

A combinatorial recommender (CR) system feeds a list of items to a user at a time in the result page, in which the user behavior is affected by both contextual information and items. The CR is formulated as a combinatorial optimization problem with the objective of maximizing the recommendation reward of the whole list. Despite its importance, it is still a challenge to build a practical CR system, due to the efficiency, dynamics, personalization requirement in online environment. In particular, we tear the problem into two sub-problems, list generation and list evaluation. Novel and practical model architectures are designed for these sub-problems aiming at jointly optimizing effectiveness and efficiency. In order to adapt to online case, a bootstrap algorithm forming an actor-critic reinforcement framework is given to explore better recommendation mode in long-term user interaction. Offline and online experiment results demonstrate the efficacy of proposed JDRec framework. JDRec has been applied in online JD recommendation, improving click through rate by 2.6% and synthetical value for the platform by 5.03%. We will publish the large-scale dataset used in this study to contribute to the research community.