# Improving Sequential Recommendation Models with an Enhanced Loss Function

arXiv: https://arxiv.org/abs/2301.00979 · HTML: https://mlsanity.com/paper/2301.00979/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 3 Jan 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/1811.00855/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Hierarchical Gating Networks for Sequential Recommendation](https://mlsanity.com/paper/1906.09217/) (2019)
- [Exploring Data Splitting Strategies for the Evaluation of Recommendation Models](https://mlsanity.com/paper/2007.13237/) (2020)
- [Memory Augmented Graph Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/1912.11730/) (2019)
- [Non-invasive Self-attention for Side Information Fusion in Sequential Recommendation](https://mlsanity.com/paper/2103.03578/) (2021)
- [A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models](https://mlsanity.com/paper/2107.13045/) (2021)
- [A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation](https://mlsanity.com/paper/2207.07483/) (2022)
- [Revisiting the Performance of iALS on Item Recommendation Benchmarks](https://mlsanity.com/paper/2110.14037/) (2021)
- [Effective and Efficient Training for Sequential Recommendation using Recency Sampling](https://mlsanity.com/paper/2207.02643/) (2022)
- [Coarse-to-Fine Sparse Sequential Recommendation](https://mlsanity.com/paper/2204.01839/) (2022)
- [Extracting Attentive Social Temporal Excitation for Sequential Recommendation](https://mlsanity.com/paper/2109.13539/) (2021)
- [Sequential Recommendation with Relation-Aware Kernelized Self-Attention](https://mlsanity.com/paper/1911.06478/) (2019)
- [ELECRec: Training Sequential Recommenders as Discriminators](https://mlsanity.com/paper/2204.02011/) (2022)

## Built on by (most cited first, 1 of 1)

- [Multi-Item-Query Attention for Stable Sequential Recommendation](https://mlsanity.com/paper/2509.24424/) (2025)

## Abstract (arXiv)

There has been a growing interest in benchmarking sequential recommendation models and reproducing/improving existing models. For example, Rendle et al. improved matrix factorization models by tuning their parameters and hyperparameters. Petrov and Macdonald developed a more efficient and effective implementation of BERT4Rec, which resolved inconsistencies in performance comparison between BERT4Rec and SASRec in previous works. In particular, BERT4Rec and SASRec share a similar network structure, with the main difference lying in their training objective/loss function. Therefore, we analyzed the advantages and disadvantages of commonly used loss functions in sequential recommendation and proposed an improved loss function that leverages their strengths. We conduct extensive experiments on two influential open-source libraries, and the results demonstrate that our improved loss function significantly enhances the performance of GRU4Rec, SASRec, SR-GNN, and S3Rec models, improving their benchmarks significantly. Furthermore, the improved SASRec benchmark outperforms BERT4Rec on the ML-1M and Beauty datasets and achieves similar results to BERT4Rec on the ML-20M and Steam datasets. We also reproduce the results of the BERT4Rec model on the Beauty dataset. Finally, we provide a comprehensive explanation of the effectiveness of our improved loss function through experiments. Our code is publicly available at https://github.com/Li-fAngyU/sequential_rec.