How Sharp and Bias-Robust is a Model? Dual Evaluation Perspectives on Knowledge Graph Completion
Not on the map. Why?
Abstract (arXiv)
Knowledge graph completion (KGC) aims to predict missing facts from the observed KG. While a number of KGC models have been studied, the evaluation of KGC still remain underexplored. In this paper, we observe that existing metrics overlook two key perspectives for KGC evaluation: (A1) predictive sharpness -- the degree of strictness in evaluating an individual prediction, and (A2) popularity-bias robustness -- the ability to predict low-popularity entities. Toward reflecting both perspectives, we propose a novel evaluation framework (PROBE), which consists of a rank transformer (RT) estimating the score of each prediction based on a required level of predictive sharpness and a rank aggregator (RA) aggregating all the scores in a popularity-aware manner. Experiments on real-world KGs reveal that existing metrics tend to over- or under-estimate the accuracy of KGC models, whereas PROBE yields a comprehensive understanding of KGC models and reliable evaluation results.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 6 Dec 2025.
Builds on
16 of 32 references.
References · 16
- 2016Complex Embeddings for Simple Link Predictioncited by 897
- 2017Convolutional 2D Knowledge Graph Embeddingscited by 729
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
- 2020A Survey on Knowledge Graph-Based Recommender Systemscited by 170
- 2018DKN: Deep Knowledge-Aware Network for News Recommendationcited by 137
- 2020RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphscited by 117
- 2018One-Shot Relational Learning for Knowledge Graphscited by 78
- 2019A Re-evaluation of Knowledge Graph Completion Methodscited by 73
- 2019Probabilistic Logic Neural Networks for Reasoningcited by 69
- 2021A Review of Biomedical Datasets Relating to Drug Discovery: A Knowledge Graph Perspectivecited by 37
- 2020Drug Repurposing for COVID-19 via Knowledge Graph Completioncited by 30
- 2019Tackling Long-Tailed Relations and Uncommon Entities in Knowledge Graph Completioncited by 19
4 more
- 2022Rethinking Knowledge Graph Evaluation Under the Open-World Assumptioncited by 17
- 2022A Unified Framework for Rank-based Evaluation Metrics for Link Prediction in Knowledge Graphscited by 14
- 2023KHAN: Knowledge-Aware Hierarchical Attention Networks for Accurate Political Stance Predictioncited by 8
- 2023SAGE: A Storage-Based Approach for Scalable and Efficient Sparse Generalized Matrix-Matrix Multiplicationcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.