# Effective and Efficient Training for Sequential Recommendation using Recency Sampling

arXiv: https://arxiv.org/abs/2207.02643 · HTML: https://mlsanity.com/paper/2207.02643/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation](https://mlsanity.com/paper/2110.05730/) (2021)
- [A Simple Convolutional Generative Network for Next Item Recommendation](https://mlsanity.com/paper/1808.05163/) (2018)
- [Hierarchical Gating Networks for Sequential Recommendation](https://mlsanity.com/paper/1906.09217/) (2019)
- [Sequence-Aware Recommender Systems](https://mlsanity.com/paper/1802.08452/) (2018)
- [Dynamic Graph Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/2104.07368/) (2021)
- [Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer](https://mlsanity.com/paper/2105.00522/) (2021)
- [Metadata Embeddings for User and Item Cold-start Recommendations](https://mlsanity.com/paper/1507.08439/) (2015)
- [A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models](https://mlsanity.com/paper/2107.13045/) (2021)
- [GAG: Global Attributed Graph Neural Network for Streaming Session-based Recommendation](https://mlsanity.com/paper/2007.02747/) (2020)
- [Using Temporal Data for Making Recommendations](https://mlsanity.com/paper/1301.2320/) (2013)
- [Exploiting Cross-Session Information for Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/2107.00852/) (2021)
- [MEANTIME: Mixture of Attention Mechanisms with Multi-temporal Embeddings for Sequential Recommendation](https://mlsanity.com/paper/2008.08273/) (2020)
- [Memory Augmented Multi-Instance Contrastive Predictive Coding for Sequential Recommendation](https://mlsanity.com/paper/2109.00368/) (2021)
- [Exploiting Positional Information for Session-based Recommendation](https://mlsanity.com/paper/2107.00846/) (2021)
- [KATRec: Knowledge Aware aTtentive Sequential Recommendations](https://mlsanity.com/paper/2012.03323/) (2020)
- [Attention-based neural re-ranking approach for next city in trip recommendations](https://mlsanity.com/paper/2103.12475/) (2021)
- [MOI-Mixer: Improving MLP-Mixer with Multi Order Interactions in Sequential Recommendation](https://mlsanity.com/paper/2108.07505/) (2021)
- [New Insights into Metric Optimization for Ranking-based Recommendation](https://mlsanity.com/paper/2106.02545/) (2021)

## Built on by (most cited first, 13 of 13)

- [Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?](https://mlsanity.com/paper/2309.07602/) (2023)
- [gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling](https://mlsanity.com/paper/2308.07192/) (2023)
- [A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation](https://mlsanity.com/paper/2207.07483/) (2022)
- [Attention Mixtures for Time-Aware Sequential Recommendation](https://mlsanity.com/paper/2304.08158/) (2023)
- [Integrating Item Relevance in Training Loss for Sequential Recommender Systems](https://mlsanity.com/paper/2305.10824/) (2023)
- [A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems](https://mlsanity.com/paper/2406.17335/) (2024)
- [Investigating the Robustness of Sequential Recommender Systems Against Training Data Perturbations](https://mlsanity.com/paper/2307.13165/) (2023)
- [Proxy-based Item Representation for Attribute and Context-aware Recommendation](https://mlsanity.com/paper/2312.06145/) (2023)
- [Improving Sequential Recommendation Models with an Enhanced Loss Function](https://mlsanity.com/paper/2301.00979/) (2023)
- [Dynamic Collaborative Filtering for Matrix- and Tensor-based Recommender Systems](https://mlsanity.com/paper/2312.10064/) (2023)
- [Non-autoregressive Personalized Bundle Generation](https://mlsanity.com/paper/2406.06925/) (2024)
- [DeBaTeR: Denoising Bipartite Temporal Graph for Recommendation](https://mlsanity.com/paper/2411.09181/) (2024)
- [Multi-Item-Query Attention for Stable Sequential Recommendation](https://mlsanity.com/paper/2509.24424/) (2025)

## Abstract (arXiv)

Many modern sequential recommender systems use deep neural networks, which can effectively estimate the relevance of items but require a lot of time to train. Slow training increases expenses, hinders product development timescales and prevents the model from being regularly updated to adapt to changing user preferences. Training such sequential models involves appropriately sampling past user interactions to create a realistic training objective. The existing training objectives have limitations. For instance, next item prediction never uses the beginning of the sequence as a learning target, thereby potentially discarding valuable data. On the other hand, the item masking used by BERT4Rec is only weakly related to the goal of the sequential recommendation; therefore, it requires much more time to obtain an effective model. Hence, we propose a novel Recency-based Sampling of Sequences training objective that addresses both limitations. We apply our method to various recent and state-of-the-art model architectures - such as GRU4Rec, Caser, and SASRec. We show that the models enhanced with our method can achieve performances exceeding or very close to stateof-the-art BERT4Rec, but with much less training time.