# Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?

arXiv: https://arxiv.org/abs/2309.07602 · HTML: https://mlsanity.com/paper/2309.07602/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 14 Sep 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation](https://mlsanity.com/paper/2110.05730/) (2021)
- [Yelp Dataset Challenge: Review Rating Prediction](https://mlsanity.com/paper/1605.05362/) (2016)
- [Translation-based Recommendation](https://mlsanity.com/paper/1707.02410/) (2017)
- [Contrastive Self-supervised Sequential Recommendation with Robust Augmentation](https://mlsanity.com/paper/2108.06479/) (2021)
- [Non-invasive Self-attention for Side Information Fusion in Sequential Recommendation](https://mlsanity.com/paper/2103.03578/) (2021)
- [A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models](https://mlsanity.com/paper/2107.13045/) (2021)
- [Lightweight Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/2108.11333/) (2021)
- [A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation](https://mlsanity.com/paper/2207.07483/) (2022)
- [Denoising Self-attentive Sequential Recommendation](https://mlsanity.com/paper/2212.04120/) (2022)
- [Effective and Efficient Training for Sequential Recommendation using Recency Sampling](https://mlsanity.com/paper/2207.02643/) (2022)
- [Contrastive Learning with Bidirectional Transformers for Sequential Recommendation](https://mlsanity.com/paper/2208.03895/) (2022)

## Built on by (most cited first, 30 of 30)

- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [SLMRec: Distilling Large Language Models into Small for Sequential Recommendation](https://mlsanity.com/paper/2405.17890/) (2024)
- [Does It Look Sequential? An Analysis of Datasets for Evaluation of Sequential Recommendations](https://mlsanity.com/paper/2408.12008/) (2024)
- [Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs](https://mlsanity.com/paper/2409.18721/) (2024)
- [Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenders](https://mlsanity.com/paper/2507.16289/) (2025)
- [RECE: Reduced Cross-Entropy Loss for Large-Catalogue Sequential Recommenders](https://mlsanity.com/paper/2408.02354/) (2024)
- [Preference Diffusion for Recommendation](https://mlsanity.com/paper/2410.13117/) (2024)
- [Autoregressive Generation Strategies for Top-K Sequential Recommendations](https://mlsanity.com/paper/2409.17730/) (2024)
- [SimCE: Simplifying Cross-Entropy Loss for Collaborative Filtering](https://mlsanity.com/paper/2406.16170/) (2024)
- [Unlocking the Power of Diffusion Models in Sequential Recommendation: A Simple and Effective Approach](https://mlsanity.com/paper/2505.19544/) (2025)
- [Unified Representation Learning for Multi-Intent Diversity and Behavioral Uncertainty in Recommender Systems](https://mlsanity.com/paper/2509.04694/) (2025)
- [RePlay: a Recommendation Framework for Experimentation and Production Use](https://mlsanity.com/paper/2409.07272/) (2024)
- [SessionRec: Next Session Prediction Paradigm For Generative Sequential Recommendation](https://mlsanity.com/paper/2502.10157/) (2025)
- [Revisiting scalable sequential recommendation with Multi-Embedding Approach and Mixture-of-Experts](https://mlsanity.com/paper/2510.25285/) (2025)
- [To Copy, or not to Copy; That is a Critical Issue of the Output Softmax Layer in Neural Sequential Recommenders](https://mlsanity.com/paper/2310.14079/) (2023)
- [Recommendation Is a Dish Better Served Warm](https://mlsanity.com/paper/2508.07856/) (2025)
- [End-to-End Graph-Sequential Representation Learning for Accurate Recommendations](https://mlsanity.com/paper/2403.00895/) (2024)
- [Calibration-Disentangled Learning and Relevance-Prioritized Reranking for Calibrated Sequential Recommendation](https://mlsanity.com/paper/2408.02156/) (2024)
- [Let It Go? Not Quite: Addressing Item Cold Start in Sequential Recommendations with Content-Based Initialization](https://mlsanity.com/paper/2507.19473/) (2025)
- [Exploiting Preferences in Loss Functions for Sequential Recommendation via Weak Transitivity](https://mlsanity.com/paper/2408.00326/) (2024)
- [Looking around you: external information enhances representations for event sequences](https://mlsanity.com/paper/2502.10205/) (2025)
- [Compositions of Variant Experts for Integrating Short-Term and Long-Term Preferences](https://mlsanity.com/paper/2506.23170/) (2025)
- [Closing the Performance Gap in Generative Recommenders with Collaborative Tokenization and Efficient Modeling](https://mlsanity.com/paper/2508.14910/) (2025)
- [Multi-Item-Query Attention for Stable Sequential Recommendation](https://mlsanity.com/paper/2509.24424/) (2025)
- [Position-Aware Sequential Attention for Accurate Next Item Recommendations](https://mlsanity.com/paper/2602.21052/) (2026)
- [Cross-Representation Knowledge Transfer for Improved Sequential Recommendations](https://mlsanity.com/paper/2602.23471/) (2026)
- [Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation](https://mlsanity.com/paper/2604.21536/) (2026)
- [Batch Size or Negatives? A Selection Rule for Memory-Constrained Recommender Training](https://mlsanity.com/paper/2608.11061/) (2026)
- [Residual Dominance as a Structural Account of Last-Item Reliance in Causal Self-Attention Recommenders](https://mlsanity.com/paper/2608.14021/) (2026)
- [Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation](https://mlsanity.com/paper/2608.24079/) (2026)

## Abstract (arXiv)

Recently sequential recommendations and next-item prediction task has become increasingly popular in the field of recommender systems. Currently, two state-of-the-art baselines are Transformer-based models SASRec and BERT4Rec. Over the past few years, there have been quite a few publications comparing these two algorithms and proposing new state-of-the-art models. In most of the publications, BERT4Rec achieves better performance than SASRec. But BERT4Rec uses cross-entropy over softmax for all items, while SASRec uses negative sampling and calculates binary cross-entropy loss for one positive and one negative item. In our work, we show that if both models are trained with the same loss, which is used by BERT4Rec, then SASRec will significantly outperform BERT4Rec both in terms of quality and training speed. In addition, we show that SASRec could be effectively trained with negative sampling and still outperform BERT4Rec, but the number of negative examples should be much larger than one.