# Generator and Critic: A Deep Reinforcement Learning Approach for Slate Re-ranking in E-commerce

arXiv: https://arxiv.org/abs/2005.12206 · HTML: https://mlsanity.com/paper/2005.12206/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2020.

## Built on by (most cited first, 7 of 7)

- [Cross DQN: Cross Deep Q Network for Ads Allocation in Feed](https://mlsanity.com/paper/2109.04353/) (2021)
- [Discrete Conditional Diffusion for Reranking in Recommendation](https://mlsanity.com/paper/2308.06982/) (2023)
- [Variation Control and Evaluation for Generative SlateRecommendations](https://mlsanity.com/paper/2102.13302/) (2021)
- [FRAMM: Fair Ranking with Missing Modalities for Clinical Trial Site Selection](https://mlsanity.com/paper/2305.19407/) (2023)
- [Learning List-wise Representation in Reinforcement Learning for Ads Allocation with Multiple Auxiliary Tasks](https://mlsanity.com/paper/2204.00888/) (2022)
- [JDRec: Practical Actor-Critic Framework for Online Combinatorial Recommender System](https://mlsanity.com/paper/2207.13311/) (2022)
- [FlashEvaluator: Expanding Search Space with Parallel Sequence-Level Evaluation](https://mlsanity.com/paper/2603.02565/) (2026)

## Abstract (arXiv)

The slate re-ranking problem considers the mutual influences between items to improve user satisfaction in e-commerce, compared with the point-wise ranking. Previous works either directly rank items by an end to end model, or rank items by a score function that trades-off the point-wise score and the diversity between items. However, there are two main existing challenges that are not well studied: (1) the evaluation of the slate is hard due to the complex mutual influences between items of one slate; (2) even given the optimal evaluation, searching the optimal slate is challenging as the action space is exponentially large. In this paper, we present a novel Generator and Critic slate re-ranking approach, where the Critic evaluates the slate and the Generator ranks the items by the reinforcement learning approach. We propose a Full Slate Critic (FSC) model that considers the real impressed items and avoids the impressed bias of existing models. For the Generator, to tackle the problem of large action space, we propose a new exploration reinforcement learning algorithm, called PPO-Exploration. Experimental results show that the FSC model significantly outperforms the state of the art slate evaluation methods, and the PPO-Exploration algorithm outperforms the existing reinforcement learning methods substantially. The Generator and Critic approach improves both the slate efficiency(4% gmv and 5% number of orders) and diversity in live experiments on one of the largest e-commerce websites in the world.