# How Sharp and Bias-Robust is a Model? Dual Evaluation Perspectives on Knowledge Graph Completion

arXiv: https://arxiv.org/abs/2512.06296 · HTML: https://mlsanity.com/paper/2512.06296/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 6 Dec 2025.

## Builds on (most cited first, 16 of 16)

- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [A Survey on Knowledge Graph-Based Recommender Systems](https://mlsanity.com/paper/2003.00911/) (2020)
- [DKN: Deep Knowledge-Aware Network for News Recommendation](https://mlsanity.com/paper/1801.08284/) (2018)
- [RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs](https://mlsanity.com/paper/2010.04029/) (2020)
- [One-Shot Relational Learning for Knowledge Graphs](https://mlsanity.com/paper/1808.09040/) (2018)
- [A Re-evaluation of Knowledge Graph Completion Methods](https://mlsanity.com/paper/1911.03903/) (2019)
- [Probabilistic Logic Neural Networks for Reasoning](https://mlsanity.com/paper/1906.08495/) (2019)
- [A Review of Biomedical Datasets Relating to Drug Discovery: A Knowledge Graph Perspective](https://mlsanity.com/paper/2102.10062/) (2021)
- [Drug Repurposing for COVID-19 via Knowledge Graph Completion](https://mlsanity.com/paper/2010.09600/) (2020)
- [Tackling Long-Tailed Relations and Uncommon Entities in Knowledge Graph Completion](https://mlsanity.com/paper/1909.11359/) (2019)
- [Rethinking Knowledge Graph Evaluation Under the Open-World Assumption](https://mlsanity.com/paper/2209.08858/) (2022)
- [A Unified Framework for Rank-based Evaluation Metrics for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/2203.07544/) (2022)
- [KHAN: Knowledge-Aware Hierarchical Attention Networks for Accurate Political Stance Prediction](https://mlsanity.com/paper/2302.12126/) (2023)
- [SAGE: A Storage-Based Approach for Scalable and Efficient Sparse Generalized Matrix-Matrix Multiplication](https://mlsanity.com/paper/2308.13626/) (2023)

## Built on by (most cited first, 1 of 1)

- [When Metrics Disagree: A Meta-Analysis of Knowledge-Graph-Completion Model Benchmarking](https://mlsanity.com/paper/2606.10287/) (2026)

## Abstract (arXiv)

Knowledge graph completion (KGC) aims to predict missing facts from the observed KG. While a number of KGC models have been studied, the evaluation of KGC still remain underexplored. In this paper, we observe that existing metrics overlook two key perspectives for KGC evaluation: (A1) predictive sharpness -- the degree of strictness in evaluating an individual prediction, and (A2) popularity-bias robustness -- the ability to predict low-popularity entities. Toward reflecting both perspectives, we propose a novel evaluation framework (PROBE), which consists of a rank transformer (RT) estimating the score of each prediction based on a required level of predictive sharpness and a rank aggregator (RA) aggregating all the scores in a popularity-aware manner. Experiments on real-world KGs reveal that existing metrics tend to over- or under-estimate the accuracy of KGC models, whereas PROBE yields a comprehensive understanding of KGC models and reliable evaluation results.