Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven Analysis

arXiv 2308.00404 ↗First posted 1 Aug 2023cs.IR cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

The success of graph neural network-based models (GNNs) has significantly advanced recommender systems by effectively modeling users and items as a bipartite, undirected graph. However, many original graph-based works often adopt results from baseline papers without verifying their validity for the specific configuration under analysis. Our work addresses this issue by focusing on the replicability of results. We present a code that successfully replicates results from six popular and recent graph recommendation models (NGCF, DGCF, LightGCN, SGL, UltraGCN, and GFCF) on three common benchmark datasets (Gowalla, Yelp 2018, and Amazon Book). Additionally, we compare these graph models with traditional collaborative filtering models that historically performed well in offline evaluations. Furthermore, we extend our study to two new datasets (Allrecipes and BookCrossing) that lack established setups in existing literature. As the performance on these datasets differs from the previous benchmarks, we analyze the impact of specific dataset characteristics on recommendation accuracy. By investigating the information flow from users' neighborhoods, we aim to identify which models are influenced by intrinsic features in the dataset structure. The code to reproduce our experiments is available at: https://github.com/sisinflab/Graph-RSs-Reproducibility.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 56 of its 100 references that are in the map. First posted on arXiv 1 Aug 2023.

Builds on

56 of 100 references.

References · 56

2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
2017Graph Attention Networkscited by 8,854
2017Neural Message Passing for Quantum Chemistrycited by 3,460
2020Supervised Contrastive Learningcited by 2,803
2016node2vec: Scalable Feature Learning for Networkscited by 1,913
2014DeepWalk: Online Learning of Social Representationscited by 1,543
2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,134
2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
2017Neural Collaborative Filteringcited by 1,108
2019Neural Graph Collaborative Filteringcited by 660
2015LINE: Large-scale Information Network Embeddingcited by 582

38 more

2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 517
2020Graph Neural Networks in Recommender Systems: A Surveycited by 493
2019KGAT: Knowledge Graph Attention Network for Recommendationcited by 342
2018Variational Autoencoders for Collaborative Filteringcited by 308
2020Self-supervised Graph Learning for Recommendationcited by 305
2019Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approachescited by 181
2021Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendationcited by 173
2020Revisiting Graph based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approachcited by 119
2020Disentangled Graph Collaborative Filteringcited by 118
2019Learning Disentangled Representations for Recommendationcited by 110
2021Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendationcited by 101
2019Embarrassingly Shallow Autoencoders for Sparse Datacited by 93
2021UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendationcited by 87
2022Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learningcited by 86
2015Modeling User Exposure in Recommendationcited by 69
2022Knowledge Graph Contrastive Learning for Recommendationcited by 69
2019A Troubling Analysis of Reproducibility and Progress in Recommender Systems Researchcited by 65
2021SimpleX: A Simple and Strong Baseline for Collaborative Filteringcited by 65
2022Hypergraph Contrastive Collaborative Filteringcited by 63
2022BARS: Towards Open Benchmarking for Recommender Systemscited by 54
2021Graph Trend Filtering Networks for Recommendationscited by 53
2021How Powerful is Graph Convolution for Recommendation?cited by 50
2021Interest-aware Message-Passing GCN for Recommendationcited by 39
2022Contrastive Meta Learning with Behavior Multiplicity for Recommendationcited by 37
2018Spectral Collaborative Filteringcited by 34
2022HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularizationcited by 34
2020Graph Convolutional Network for Recommendation with Low-pass Collaborative Filterscited by 32
2018Collaborative Memory Network for Recommendation Systemscited by 30
2020Bipartite Graph Embedding via Mutual Information Maximizationcited by 29
2023Personalized Graph Signal Processing for Collaborative Filteringcited by 29
2021Elliot: a Comprehensive and Rigorous Framework for Reproducible Recommender Systems Evaluationcited by 28
2022Self-Guided Learning to Denoise for Robust Recommendationcited by 24
2023On the Variance of Neural Network Training with respect to Test Sets and Distributionscited by 20
2020Sampler Design for Implicit Feedback Data by Noisy-label Robust Learningcited by 18
2022SVD-GCN: A Simplified Graph Convolution Paradigm for Recommendationcited by 16
2021Graph Collaborative Reasoningcited by 14
2022Investigating Accuracy-Novelty Performance for Graph-based Collaborative Filteringcited by 13

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.