# Generative Sequential Recommendation with GPTRec

arXiv: https://arxiv.org/abs/2306.11114 · HTML: https://mlsanity.com/paper/2306.11114/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2023.

## Built on by (most cited first, 21 of 21)

- [ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation](https://mlsanity.com/paper/2308.11131/) (2023)
- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [Large Language Models for Generative Recommendation: A Survey and Visionary Discussions](https://mlsanity.com/paper/2309.01157/) (2023)
- [gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling](https://mlsanity.com/paper/2308.07192/) (2023)
- [ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation](https://mlsanity.com/paper/2502.13581/) (2025)
- [PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Process](https://mlsanity.com/paper/2507.04607/) (2025)
- [Exploring the Impact of Large Language Models on Recommender Systems: An Extensive Review](https://mlsanity.com/paper/2402.18590/) (2024)
- [A Survey on Large Language Models for Personalized and Explainable Recommendations](https://mlsanity.com/paper/2311.12338/) (2023)
- [KVDirect: Distributed Disaggregated LLM Inference](https://mlsanity.com/paper/2501.14743/) (2024)
- [Preference Discerning with LLM-Enhanced Generative Retrieval](https://mlsanity.com/paper/2412.08604/) (2024)
- [PAP-REC: Personalized Automatic Prompt for Recommendation Language Model](https://mlsanity.com/paper/2402.00284/) (2024)
- [GRACE: Generative Recommendation via Journey-Aware Sparse Attention on Chain-of-Thought Tokenization](https://mlsanity.com/paper/2507.14758/) (2025)
- [Large Language Models Make Sample-Efficient Recommender Systems](https://mlsanity.com/paper/2406.02368/) (2024)
- [CherryRec: Enhancing News Recommendation Quality via LLM-driven Framework](https://mlsanity.com/paper/2406.12243/) (2024)
- [Hyperbolic Residual Quantization: Discrete Representations for Data with Latent Hierarchies](https://mlsanity.com/paper/2505.12404/) (2025)
- [Learning Variable-Length Tokenization for Generative Recommendation](https://mlsanity.com/paper/2605.17779/) (2026)
- [Pivoting Retail Supply Chain with Deep Generative Techniques: Taxonomy, Survey and Insights](https://mlsanity.com/paper/2403.00861/) (2024)
- [Inductive Transfer Learning for Graph-Based Recommenders](https://mlsanity.com/paper/2510.22799/) (2025)
- [GRAB: An LLM-Inspired Sequence-First Click-Through Rate Prediction Modeling Paradigm](https://mlsanity.com/paper/2602.01865/) (2026)
- [Jump Start or False Start? A Theoretical and Empirical Evaluation of LLM-initialized Bandits](https://mlsanity.com/paper/2604.02527/) (2026)
- [Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems](https://mlsanity.com/paper/2607.19357/) (2026)

## Abstract (arXiv)

Sequential recommendation is an important recommendation task that aims to predict the next item in a sequence. Recently, adaptations of language models, particularly Transformer-based models such as SASRec and BERT4Rec, have achieved state-of-the-art results in sequential recommendation. In these models, item ids replace tokens in the original language models. However, this approach has limitations. First, the vocabulary of item ids may be many times larger than in language models. Second, the classical Top-K recommendation approach used by these models may not be optimal for complex recommendation objectives, including auxiliary objectives such as diversity, coverage or coherence. Recent progress in generative language models inspires us to revisit generative approaches to address these challenges. This paper presents the GPTRec sequential recommendation model, which is based on the GPT-2 architecture. GPTRec can address large vocabulary issues by splitting item ids into sub-id tokens using a novel SVD Tokenisation algorithm based on quantised item embeddings from an SVD decomposition of the user-item interaction matrix. The paper also presents a novel Next-K recommendation strategy, which generates recommendations item-by-item, considering already recommended items. The Next-K strategy can be used for producing complex interdependent recommendation lists. We experiment with GPTRec on the MovieLens-1M dataset and show that using sub-item tokenisation GPTRec can match the quality of SASRec while reducing the embedding table by 40%. We also show that the recommendations generated by GPTRec on MovieLens-1M using the Next-K recommendation strategy match the quality of SASRec in terms of NDCG@10, meaning that the model can serve as a strong starting point for future research.