# Generating Negative Samples for Sequential Recommendation

arXiv: https://arxiv.org/abs/2208.03645 · HTML: https://mlsanity.com/paper/2208.03645/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 7 Aug 2022.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation](https://mlsanity.com/paper/2110.05730/) (2021)
- [Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency](https://mlsanity.com/paper/1809.01812/) (2018)
- [Intent Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2202.02519/) (2022)
- [IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models](https://mlsanity.com/paper/1705.10513/) (2017)
- [Understanding Negative Sampling in Graph Representation Learning](https://mlsanity.com/paper/2005.09863/) (2020)
- [KBGAN: Adversarial Learning for Knowledge Graph Embeddings](https://mlsanity.com/paper/1711.04071/) (2017)
- [Contrastive Self-supervised Sequential Recommendation with Robust Augmentation](https://mlsanity.com/paper/2108.06479/) (2021)
- [Understanding Hard Negatives in Noise Contrastive Estimation](https://mlsanity.com/paper/2104.06245/) (2021)
- [Adaptive Sampled Softmax with Kernel Based Sampling](https://mlsanity.com/paper/1712.00527/) (2017)
- [Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems](https://mlsanity.com/paper/2005.12964/) (2020)
- [Adversarial Sampling and Training for Semi-Supervised Information Retrieval](https://mlsanity.com/paper/1811.04155/) (2018)
- [Lightweight Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/2108.11333/) (2021)
- [Memory Augmented Multi-Instance Contrastive Predictive Coding for Sequential Recommendation](https://mlsanity.com/paper/2109.00368/) (2021)

## Built on by (most cited first, 3 of 3)

- [RECE: Reduced Cross-Entropy Loss for Large-Catalogue Sequential Recommenders](https://mlsanity.com/paper/2408.02354/) (2024)
- [Evaluating Performance and Bias of Negative Sampling in Large-Scale Sequential Recommendation Models](https://mlsanity.com/paper/2410.17276/) (2024)
- [Exploiting Preferences in Loss Functions for Sequential Recommendation via Weak Transitivity](https://mlsanity.com/paper/2408.00326/) (2024)

## Abstract (arXiv)

To make Sequential Recommendation (SR) successful, recent works focus on designing effective sequential encoders, fusing side information, and mining extra positive self-supervision signals. The strategy of sampling negative items at each time step is less explored. Due to the dynamics of users' interests and model updates during training, considering randomly sampled items from a user's non-interacted item set as negatives can be uninformative. As a result, the model will inaccurately learn user preferences toward items. Identifying informative negatives is challenging because informative negative items are tied with both dynamically changed interests and model parameters (and sampling process should also be efficient). To this end, we propose to Generate Negative Samples (items) for SR (GenNi). A negative item is sampled at each time step based on the current SR model's learned user preferences toward items. An efficient implementation is proposed to further accelerate the generation process, making it scalable to large-scale recommendation tasks. Extensive experiments on four public datasets verify the importance of providing high-quality negative samples for SR and demonstrate the effectiveness and efficiency of GenNi.