# Performance Comparison of Session-based Recommendation Algorithms based on GNNs

arXiv: https://arxiv.org/abs/2312.16695 · HTML: https://mlsanity.com/paper/2312.16695/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 27 Dec 2023.

## Builds on (most cited first, 22 of 22)

- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Representation Learning on Graphs: Methods and Applications](https://mlsanity.com/paper/1709.05584/) (2017)
- [Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/1811.00855/) (2018)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Global Context Enhanced Graph Neural Networks for Session-based Recommendation](https://mlsanity.com/paper/2106.05081/) (2021)
- [Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision](https://mlsanity.com/paper/2109.08203/) (2021)
- [Session-based Social Recommendation via Dynamic Graph Attention Networks](https://mlsanity.com/paper/1902.09362/) (2019)
- [A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research](https://mlsanity.com/paper/1911.07698/) (2019)
- [Neural Embeddings of Graphs in Hyperbolic Space](https://mlsanity.com/paper/1705.10359/) (2017)
- [On the Difficulty of Evaluating Baselines: A Study on Recommender Systems](https://mlsanity.com/paper/1905.01395/) (2019)
- [Self-Supervised Graph Co-Training for Session-based Recommendation](https://mlsanity.com/paper/2108.10560/) (2021)
- [Evaluation of Session-based Recommendation Algorithms](https://mlsanity.com/paper/1803.09587/) (2018)
- [CORE: Simple and Effective Session-based Recommendation within Consistent Representation Space](https://mlsanity.com/paper/2204.11067/) (2022)
- [TAGNN: Target Attentive Graph Neural Networks for Session-based Recommendation](https://mlsanity.com/paper/2005.02844/) (2020)
- [Heterogeneous Global Graph Neural Networks for Personalized Session-based Recommendation](https://mlsanity.com/paper/2107.03813/) (2021)
- [Revisiting the Performance of iALS on Item Recommendation Benchmarks](https://mlsanity.com/paper/2110.14037/) (2021)
- [Future Data Helps Training: Modeling Future Contexts for Session-based Recommendation](https://mlsanity.com/paper/1906.04473/) (2019)
- [Empirical Analysis of Session-Based Recommendation Algorithms](https://mlsanity.com/paper/1910.12781/) (2019)
- [Top-N Recommendation Algorithms: A Quest for the State-of-the-Art](https://mlsanity.com/paper/2203.01155/) (2022)
- [The Effect of Third Party Implementations on Reproducibility](https://mlsanity.com/paper/2307.14956/) (2023)

## Built on by (most cited first, 1 of 1)

- [Recommender Systems for Good (RS4Good): Survey of Use Cases and a Call to Action for Research that Matters](https://mlsanity.com/paper/2411.16645/) (2024)

## Abstract (arXiv)

In session-based recommendation settings, a recommender system has no access to long-term user profiles and thus has to base its suggestions on the user interactions that are observed in an ongoing session. Since such sessions can consist of only a small set of interactions, various approaches based on Graph Neural Networks (GNN) were recently proposed, as they allow us to integrate various types of side information about the items in a natural way. Unfortunately, a variety of evaluation settings are used in the literature, e.g., in terms of protocols, metrics and baselines, making it difficult to assess what represents the state of the art. In this work, we present the results of an evaluation of eight recent GNN-based approaches that were published in high-quality outlets. For a fair comparison, all models are systematically tuned and tested under identical conditions using three common datasets. We furthermore include k-nearest-neighbor and sequential rules-based models as baselines, as such models have previously exhibited competitive performance results for similar settings. To our surprise, the evaluation showed that the simple models outperform all recent GNN models in terms of the Mean Reciprocal Rank, which we used as an optimization criterion, and were only outperformed in three cases in terms of the Hit Rate. Additional analyses furthermore reveal that several other factors that are often not deeply discussed in papers, e.g., random seeds, can markedly impact the performance of GNN-based models. Our results therefore (a) point to continuing issues in the community in terms of research methodology and (b) indicate that there is ample room for improvement in session-based recommendation.