# gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling

arXiv: https://arxiv.org/abs/2308.07192 · HTML: https://mlsanity.com/paper/2308.07192/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 14 Aug 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Mitigating Neural Network Overconfidence with Logit Normalization](https://mlsanity.com/paper/2205.09310/) (2022)
- [Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation](https://mlsanity.com/paper/2110.05730/) (2021)
- [A Simple Convolutional Generative Network for Next Item Recommendation](https://mlsanity.com/paper/1808.05163/) (2018)
- [On Using Very Large Target Vocabulary for Neural Machine Translation](https://mlsanity.com/paper/1412.2007/) (2014)
- [On the Effectiveness of Sampled Softmax Loss for Item Recommendation](https://mlsanity.com/paper/2201.02327/) (2022)
- [A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models](https://mlsanity.com/paper/2107.13045/) (2021)
- [A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation](https://mlsanity.com/paper/2207.07483/) (2022)
- [Generative Sequential Recommendation with GPTRec](https://mlsanity.com/paper/2306.11114/) (2023)
- [A Re-visit of the Popularity Baseline in Recommender Systems](https://mlsanity.com/paper/2005.13829/) (2020)
- [Item Recommendation from Implicit Feedback](https://mlsanity.com/paper/2101.08769/) (2021)
- [Effective and Efficient Training for Sequential Recommendation using Recency Sampling](https://mlsanity.com/paper/2207.02643/) (2022)
- [Contrastive Learning with Bidirectional Transformers for Sequential Recommendation](https://mlsanity.com/paper/2208.03895/) (2022)

## Built on by (most cited first, 29 of 29)

- [Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation](https://mlsanity.com/paper/2503.22675/) (2025)
- [Evaluating ChatGPT as a Recommender System: A Rigorous Approach](https://mlsanity.com/paper/2309.03613/) (2023)
- [SLMRec: Distilling Large Language Models into Small for Sequential Recommendation](https://mlsanity.com/paper/2405.17890/) (2024)
- [Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs](https://mlsanity.com/paper/2409.18721/) (2024)
- [Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenders](https://mlsanity.com/paper/2507.16289/) (2025)
- [End-to-end Learnable Clustering for Intent Learning in Recommendation](https://mlsanity.com/paper/2401.05975/) (2024)
- [RECE: Reduced Cross-Entropy Loss for Large-Catalogue Sequential Recommenders](https://mlsanity.com/paper/2408.02354/) (2024)
- [Captions Speak Louder than Images: Generalizing Foundation Models for E-commerce from High-quality Multimodal Instruction Data](https://mlsanity.com/paper/2410.17337/) (2024)
- [Autoregressive Generation Strategies for Top-K Sequential Recommendations](https://mlsanity.com/paper/2409.17730/) (2024)
- [CLCE: An Approach to Refining Cross-Entropy and Contrastive Learning for Optimized Learning Fusion](https://mlsanity.com/paper/2402.14551/) (2024)
- [Shallow Cross-Encoders for Low-Latency Retrieval](https://mlsanity.com/paper/2403.20222/) (2024)
- [Evaluating Performance and Bias of Negative Sampling in Large-Scale Sequential Recommendation Models](https://mlsanity.com/paper/2410.17276/) (2024)
- [Building a Scalable, Effective, and Steerable Search and Ranking Platform](https://mlsanity.com/paper/2409.02856/) (2024)
- [RePlay: a Recommendation Framework for Experimentation and Production Use](https://mlsanity.com/paper/2409.07272/) (2024)
- [eSASRec: Enhancing Transformer-based Recommendations in a Modular Fashion](https://mlsanity.com/paper/2508.06450/) (2025)
- [Sparse Autoencoders for Sequential Recommendation Models: Interpretation and Flexible Control](https://mlsanity.com/paper/2507.12202/) (2025)
- [DenseRec: Revisiting Dense Content Embeddings for Sequential Transformer-based Recommendation](https://mlsanity.com/paper/2508.18442/) (2025)
- [Balancing Accuracy and Novelty with Sub-Item Popularity](https://mlsanity.com/paper/2508.05198/) (2025)
- [Revisiting Softmax Masking: Stop Gradient for Enhancing Stability in Replay-based Continual Learning](https://mlsanity.com/paper/2309.14808/) (2023)
- [Aligning GPTRec with Beyond-Accuracy Goals with Reinforcement Learning](https://mlsanity.com/paper/2403.04875/) (2024)
- [Negative Feedback for Music Personalization](https://mlsanity.com/paper/2406.04488/) (2024)
- [Exploiting Preferences in Loss Functions for Sequential Recommendation via Weak Transitivity](https://mlsanity.com/paper/2408.00326/) (2024)
- [Compositions of Variant Experts for Integrating Short-Term and Long-Term Preferences](https://mlsanity.com/paper/2506.23170/) (2025)
- [Does Weighting Improve Matrix Factorization for Recommender Systems?](https://mlsanity.com/paper/2510.10440/) (2025)
- [Exploring Diversity, Novelty, and Popularity Bias in ChatGPT's Recommendations](https://mlsanity.com/paper/2601.01997/) (2026)
- [Aligning Recommendations with User Popularity Preferences](https://mlsanity.com/paper/2604.01036/) (2026)
- [Batch Size or Negatives? A Selection Rule for Memory-Constrained Recommender Training](https://mlsanity.com/paper/2608.11061/) (2026)
- [Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation](https://mlsanity.com/paper/2608.24079/) (2026)
- [Balancing Trial and Reorder: A Hybrid Sequential Transformer-GBDT Ranker for On-Demand Delivery](https://mlsanity.com/paper/2609.16407/) (2026)

## Abstract (arXiv)

A large catalogue size is one of the central challenges in training recommendation models: a large number of items makes them memory and computationally inefficient to compute scores for all items during training, forcing these models to deploy negative sampling. However, negative sampling increases the proportion of positive interactions in the training data, and therefore models trained with negative sampling tend to overestimate the probabilities of positive interactions a phenomenon we call overconfidence. While the absolute values of the predicted scores or probabilities are not important for the ranking of retrieved recommendations, overconfident models may fail to estimate nuanced differences in the top-ranked items, resulting in degraded performance. In this paper, we show that overconfidence explains why the popular SASRec model underperforms when compared to BERT4Rec. This is contrary to the BERT4Rec authors explanation that the difference in performance is due to the bi-directional attention mechanism. To mitigate overconfidence, we propose a novel Generalised Binary Cross-Entropy Loss function (gBCE) and theoretically prove that it can mitigate overconfidence. We further propose the gSASRec model, an improvement over SASRec that deploys an increased number of negatives and the gBCE loss. We show through detailed experiments on three datasets that gSASRec does not exhibit the overconfidence problem. As a result, gSASRec can outperform BERT4Rec (e.g. +9.47% NDCG on the MovieLens-1M dataset), while requiring less training time (e.g. -73% training time on MovieLens-1M). Moreover, in contrast to BERT4Rec, gSASRec is suitable for large datasets that contain more than 1 million items.