# Autoregressive Generation Strategies for Top-K Sequential Recommendations

arXiv: https://arxiv.org/abs/2409.17730 · HTML: https://mlsanity.com/paper/2409.17730/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 66 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretation](https://mlsanity.com/paper/2304.03879/) (2023)
- [Yelp Dataset Challenge: Review Rating Prediction](https://mlsanity.com/paper/1605.05362/) (2016)
- [Sequence-Aware Recommender Systems](https://mlsanity.com/paper/1802.08452/) (2018)
- [Consistency of a Recurrent Language Model With Respect to Incomplete Decoding](https://mlsanity.com/paper/2002.02492/) (2020)
- [Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?](https://mlsanity.com/paper/2309.07602/) (2023)
- [gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling](https://mlsanity.com/paper/2308.07192/) (2023)
- [A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation](https://mlsanity.com/paper/2207.07483/) (2022)
- [Widespread Flaws in Offline Evaluation of Recommender Systems](https://mlsanity.com/paper/2307.14951/) (2023)
- [Personalized Bundle List Recommendation](https://mlsanity.com/paper/1904.01933/) (2019)
- [MEANTIME: Mixture of Attention Mechanisms with Multi-temporal Embeddings for Sequential Recommendation](https://mlsanity.com/paper/2008.08273/) (2020)
- [Scaling Session-Based Transformer Recommendations using Optimized Negative Sampling and Loss Functions](https://mlsanity.com/paper/2307.14906/) (2023)
- [RECE: Reduced Cross-Entropy Loss for Large-Catalogue Sequential Recommenders](https://mlsanity.com/paper/2408.02354/) (2024)
- [KATRec: Knowledge Aware aTtentive Sequential Recommendations](https://mlsanity.com/paper/2012.03323/) (2020)
- [Integrating Item Relevance in Training Loss for Sequential Recommender Systems](https://mlsanity.com/paper/2305.10824/) (2023)
- [Recurrent Neural Networks for Long and Short-Term Sequential Recommendation](https://mlsanity.com/paper/1807.09142/) (2018)

## Built on by (most cited first, 5 of 5)

- [Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenders](https://mlsanity.com/paper/2507.16289/) (2025)
- [Recommendation Is a Dish Better Served Warm](https://mlsanity.com/paper/2508.07856/) (2025)
- [Maximum Impact with Fewer Features: Efficient Feature Selection for Cold-Start Recommenders through Collaborative Importance Weighting](https://mlsanity.com/paper/2508.06455/) (2025)
- [SplitLight: An Exploratory Toolkit for Recommender Systems Datasets and Splits](https://mlsanity.com/paper/2602.19339/) (2026)
- [Jump Start or False Start? A Theoretical and Empirical Evaluation of LLM-initialized Bandits](https://mlsanity.com/paper/2604.02527/) (2026)

## Abstract (arXiv)

The goal of modern sequential recommender systems is often formulated in terms of next-item prediction. In this paper, we explore the applicability of generative transformer-based models for the Top-K sequential recommendation task, where the goal is to predict items a user is likely to interact with in the "near future". We explore commonly used autoregressive generation strategies, including greedy decoding, beam search, and temperature sampling, to evaluate their performance for the Top-K sequential recommendation task. In addition, we propose novel Reciprocal Rank Aggregation (RRA) and Relevance Aggregation (RA) generation strategies based on multi-sequence generation with temperature sampling and subsequent aggregation. Experiments on diverse datasets give valuable insights regarding commonly used strategies' applicability and show that suggested approaches improve performance on longer time horizons compared to widely-used Top-K prediction approach and single-sequence autoregressive generation strategies.