mlsanityData to 14 Oct 2026

Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?

arXiv 2309.07602 ↗First posted 14 Sep 2023cs.IR cs.AI cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

Recently sequential recommendations and next-item prediction task has become increasingly popular in the field of recommender systems. Currently, two state-of-the-art baselines are Transformer-based models SASRec and BERT4Rec. Over the past few years, there have been quite a few publications comparing these two algorithms and proposing new state-of-the-art models. In most of the publications, BERT4Rec achieves better performance than SASRec. But BERT4Rec uses cross-entropy over softmax for all items, while SASRec uses negative sampling and calculates binary cross-entropy loss for one positive and one negative item. In our work, we show that if both models are trained with the same loss, which is used by BERT4Rec, then SASRec will significantly outperform BERT4Rec both in terms of quality and training speed. In addition, we show that SASRec could be effectively trained with negative sampling and still outperform BERT4Rec, but the number of negative examples should be much larger than one.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 21 of its 36 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

21 of 36 references.

References · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  4. 2018Self-Attentive Sequential Recommendationcited by 1,184
  5. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  6. 2015Image-based Recommendations on Styles and Substitutescited by 706
  7. 2015Session-based Recommendations with Recurrent Neural Networkscited by 695
  8. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 476
  9. 2017Recurrent Neural Networks with Top-k Gains for Session-based Recommendationscited by 165
  10. 2016Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendationcited by 152
  11. 2021Contrastive Learning for Representation Degeneration Problem in Sequential Recommendationcited by 121
  12. 2016Yelp Dataset Challenge: Review Rating Predictioncited by 84
9 more
  1. 2017Translation-based Recommendationcited by 71
  2. 2021Contrastive Self-supervised Sequential Recommendation with Robust Augmentationcited by 56
  3. 2021Non-invasive Self-attention for Side Information Fusion in Sequential Recommendationcited by 39
  4. 2021A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Modelscited by 25
  5. 2021Lightweight Self-Attentive Sequential Recommendationcited by 22
  6. 2022A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendationcited by 22
  7. 2022Denoising Self-attentive Sequential Recommendationcited by 20
  8. 2022Effective and Efficient Training for Sequential Recommendation using Recency Samplingcited by 13
  9. 2022Contrastive Learning with Bidirectional Transformers for Sequential Recommendationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 12; 2025: 11; 2026: 6.

30 papers.

Sort:

By area:

  1. 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 188
  2. 2024SLMRec: Distilling Large Language Models into Small for Sequential Recommendationcited by 18
  3. 2024Does It Look Sequential? An Analysis of Datasets for Evaluation of Sequential Recommendationscited by 15
  4. 2024Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogscited by 15
  5. 2025Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenderscited by 12
  6. 2024RECE: Reduced Cross-Entropy Loss for Large-Catalogue Sequential Recommenderscited by 9
  7. 2024Preference Diffusion for Recommendationcited by 9
  8. 2024Autoregressive Generation Strategies for Top-K Sequential Recommendationscited by 5
  9. 2024SimCE: Simplifying Cross-Entropy Loss for Collaborative Filteringcited by 4
  10. 2025Unlocking the Power of Diffusion Models in Sequential Recommendation: A Simple and Effective Approachcited by 4
  11. 2025Unified Representation Learning for Multi-Intent Diversity and Behavioral Uncertainty in Recommender Systemscited by 4
  12. 2024RePlay: a Recommendation Framework for Experimentation and Production Usecited by 3
18 more
  1. 2025SessionRec: Next Session Prediction Paradigm For Generative Sequential Recommendationcited by 3
  2. 2025Revisiting scalable sequential recommendation with Multi-Embedding Approach and Mixture-of-Expertscited by 3
  3. 2023To Copy, or not to Copy; That is a Critical Issue of the Output Softmax Layer in Neural Sequential Recommenderscited by 2
  4. 2025Recommendation Is a Dish Better Served Warmcited by 2
  5. 2024End-to-End Graph-Sequential Representation Learning for Accurate Recommendationscited by 1
  6. 2024Calibration-Disentangled Learning and Relevance-Prioritized Reranking for Calibrated Sequential Recommendationcited by 1
  7. 2025Let It Go? Not Quite: Addressing Item Cold Start in Sequential Recommendations with Content-Based Initializationcited by 1
  8. 2024Exploiting Preferences in Loss Functions for Sequential Recommendation via Weak Transitivitycited by 0
  9. 2025Looking around you: external information enhances representations for event sequencescited by 0
  10. 2025Compositions of Variant Experts for Integrating Short-Term and Long-Term Preferencescited by 0
  11. 2025Closing the Performance Gap in Generative Recommenders with Collaborative Tokenization and Efficient Modelingcited by 0
  12. 2025Multi-Item-Query Attention for Stable Sequential Recommendationcited by 0
  13. 2026Position-Aware Sequential Attention for Accurate Next Item Recommendationscited by 0
  14. 2026Cross-Representation Knowledge Transfer for Improved Sequential Recommendationscited by 0
  15. 2026Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillationcited by 0
  16. 2026Batch Size or Negatives? A Selection Rule for Memory-Constrained Recommender Trainingcited by 0
  17. 2026Residual Dominance as a Structural Account of Last-Item Reliance in Causal Self-Attention Recommenderscited by 0
  18. 2026Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendationcited by 0