# KG4RecEval: Does Knowledge Graph Really Matter for Recommender Systems?

arXiv: https://arxiv.org/abs/2404.03164 · HTML: https://mlsanity.com/paper/2404.03164/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 60 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 25 of 25)

- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks](https://mlsanity.com/paper/2112.14936/) (2021)
- [A Survey on Knowledge Graph-Based Recommender Systems](https://mlsanity.com/paper/2003.00911/) (2020)
- [RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems](https://mlsanity.com/paper/1803.03467/) (2018)
- [RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms](https://mlsanity.com/paper/2011.01731/) (2020)
- [Knowledge Graph Convolutional Networks for Recommender Systems](https://mlsanity.com/paper/1904.12575/) (2019)
- [Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems](https://mlsanity.com/paper/1905.04413/) (2019)
- [Learning Intents behind Interactions with Knowledge Graph for Recommendation](https://mlsanity.com/paper/2102.07057/) (2021)
- [Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences](https://mlsanity.com/paper/1902.06236/) (2019)
- [Knowledge Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2205.00976/) (2022)
- [DiffKG: Knowledge Graph Diffusion Model for Recommendation](https://mlsanity.com/paper/2312.16890/) (2023)
- [Jointly Learning Explainable Rules for Recommendation with Knowledge Graph](https://mlsanity.com/paper/1903.03714/) (2019)
- [Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation](https://mlsanity.com/paper/1805.03352/) (2018)
- [RecBole 2.0: Towards a More Up-to-Date Recommendation Library](https://mlsanity.com/paper/2206.07351/) (2022)
- [Modeling Scale-free Graphs with Hyperbolic Geometry for Knowledge-aware Recommendation](https://mlsanity.com/paper/2108.06468/) (2021)
- [Data Poisoning Attack against Knowledge Graph Embedding](https://mlsanity.com/paper/1904.12052/) (2019)
- [Knowledge Graph Self-Supervised Rationalization for Recommendation](https://mlsanity.com/paper/2307.02759/) (2023)
- [KB4Rec: A Dataset for Linking Knowledge Bases with Recommender Systems](https://mlsanity.com/paper/1807.11141/) (2018)
- [Deep Learning on Knowledge Graph for Recommender System: A Survey](https://mlsanity.com/paper/2004.00387/) (2020)
- [DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN](https://mlsanity.com/paper/2108.11883/) (2021)
- [HAKG: Hierarchy-Aware Knowledge Gated Network for Recommendation](https://mlsanity.com/paper/2204.04959/) (2022)
- [SetRank: A Setwise Bayesian Approach for Collaborative Ranking from Implicit Feedback](https://mlsanity.com/paper/2002.09841/) (2020)
- [RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems](https://mlsanity.com/paper/2201.04399/) (2022)
- [Should Graph Neural Networks Use Features, Edges, Or Both?](https://mlsanity.com/paper/2103.06857/) (2021)
- [Recent Advances in RecBole: Extensions with more Practical Considerations](https://mlsanity.com/paper/2211.15148/) (2022)

## Built on by (most cited first, 1 of 1)

- [SEMMA: A Semantic Aware Knowledge Graph Foundation Model](https://mlsanity.com/paper/2505.20422/) (2025)

## Abstract (arXiv)

Recommender systems (RSs) are designed to provide personalized recommendations to users. Recently, knowledge graphs (KGs) have been widely introduced in RSs to improve recommendation accuracy. In this study, however, we demonstrate that RSs do not necessarily perform worse even if the KG is downgraded to the user-item interaction graph only (or removed). We propose an evaluation framework KG4RecEval to systematically evaluate how much a KG contributes to the recommendation accuracy of a KG-based RS, using our defined metric KGER (KG utilization efficiency in recommendation). We consider the scenarios where knowledge in a KG gets completely removed, randomly distorted and decreased, and also where recommendations are for cold-start users. Our extensive experiments on four commonly used datasets and a number of state-of-the-art KG-based RSs reveal that: to remove, randomly distort or decrease knowledge does not necessarily decrease recommendation accuracy, even for cold-start users. These findings inspire us to rethink how to better utilize knowledge from existing KGs, whereby we discuss and provide insights into what characteristics of datasets and KG-based RSs may help improve KG utilization efficiency. The code and supplementary material of this paper are available at: https://github.com/HotBento/KG4RecEval.