# Generative Slate Recommendation with Reinforcement Learning

arXiv: https://arxiv.org/abs/2301.08632 · HTML: https://mlsanity.com/paper/2301.08632/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 20 Jan 2023.

## Builds on (most cited first, 17 of 17)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Decoupling Representation Learning from Reinforcement Learning](https://mlsanity.com/paper/2009.08319/) (2020)
- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [PLAS: Latent Action Space for Offline Reinforcement Learning](https://mlsanity.com/paper/2011.07213/) (2020)
- [Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems](https://mlsanity.com/paper/1902.05570/) (2019)
- [Learning Action Representations for Reinforcement Learning](https://mlsanity.com/paper/1902.00183/) (2019)
- [Generative Adversarial User Model for Reinforcement Learning Based Recommendation System](https://mlsanity.com/paper/1812.10613/) (2018)
- [Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions](https://mlsanity.com/paper/2007.12986/) (2020)
- [Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation](https://mlsanity.com/paper/1911.03845/) (2019)
- [Beyond Greedy Ranking: Slate Optimization via List-CVAE](https://mlsanity.com/paper/1803.01682/) (2018)
- [Deep Reinforcement Learning with Attention for Slate Markov Decision Processes with High-Dimensional States and Actions](https://mlsanity.com/paper/1512.01124/) (2015)
- [The closed loop between opinion formation and personalised recommendations](https://mlsanity.com/paper/1809.04644/) (2018)
- [Variation Control and Evaluation for Generative SlateRecommendations](https://mlsanity.com/paper/2102.13302/) (2021)
- [Low-Dimensional State and Action Representation Learning with MDP Homomorphism Metrics](https://mlsanity.com/paper/2107.01677/) (2021)

## Built on by (most cited first, 13 of 13)

- [A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)](https://mlsanity.com/paper/2404.00579/) (2024)
- [Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstraction](https://mlsanity.com/paper/2402.02171/) (2024)
- [RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systems](https://mlsanity.com/paper/2508.00201/) (2025)
- [Time-Constrained Recommendations: Reinforcement Learning Strategies for E-Commerce](https://mlsanity.com/paper/2512.13726/) (2025)
- [Impression-Aware Recommender Systems](https://mlsanity.com/paper/2308.07857/) (2023)
- [HiGR: Industrial-Scale Hierarchical Generative Slate Recommendation Framework in Tencent](https://mlsanity.com/paper/2512.24787/) (2025)
- [PCN-Rec: Agentic Proof-Carrying Negotiation for Reliable Governance-Constrained Recommendation](https://mlsanity.com/paper/2601.09771/) (2026)
- [Aligning GPTRec with Beyond-Accuracy Goals with Reinforcement Learning](https://mlsanity.com/paper/2403.04875/) (2024)
- [Supervised Learning-enhanced Multi-Group Actor Critic for Live Stream Allocation in Feed](https://mlsanity.com/paper/2412.10381/) (2024)
- [Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion](https://mlsanity.com/paper/2603.06397/) (2026)
- [RecoAtlas: From Semantic Plausibility to Set-Level Utility in LLM Recommendation Agents](https://mlsanity.com/paper/2605.18805/) (2026)
- [Sequential Multimodal Evidence Optimization for Product Media Ranking in E-Commerce](https://mlsanity.com/paper/2608.15662/) (2026)
- [Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning](https://mlsanity.com/paper/2610.13504/) (2026)

## Abstract (arXiv)

Recent research has employed reinforcement learning (RL) algorithms to optimize long-term user engagement in recommender systems, thereby avoiding common pitfalls such as user boredom and filter bubbles. They capture the sequential and interactive nature of recommendations, and thus offer a principled way to deal with long-term rewards and avoid myopic behaviors. However, RL approaches are intractable in the slate recommendation scenario - where a list of items is recommended at each interaction turn - due to the combinatorial action space. In that setting, an action corresponds to a slate that may contain any combination of items. While previous work has proposed well-chosen decompositions of actions so as to ensure tractability, these rely on restrictive and sometimes unrealistic assumptions. Instead, in this work we propose to encode slates in a continuous, low-dimensional latent space learned by a variational auto-encoder. Then, the RL agent selects continuous actions in this latent space, which are ultimately decoded into the corresponding slates. By doing so, we are able to (i) relax assumptions required by previous work, and (ii) improve the quality of the action selection by modeling full slates instead of independent items, in particular by enabling diversity. Our experiments performed on a wide array of simulated environments confirm the effectiveness of our generative modeling of slates over baselines in practical scenarios where the restrictive assumptions underlying the baselines are lifted. Our findings suggest that representation learning using generative models is a promising direction towards generalizable RL-based slate recommendation.