# On the Ambiguity of Rank-Based Evaluation of Entity Alignment or Link Prediction Methods

arXiv: https://arxiv.org/abs/2002.06914 · HTML: https://mlsanity.com/paper/2002.06914/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Feb 2020.

## Built on by (most cited first, 13 of 13)

- [A Review of Biomedical Datasets Relating to Drug Discovery: A Knowledge Graph Perspective](https://mlsanity.com/paper/2102.10062/) (2021)
- [An Open Challenge for Inductive Link Prediction on Knowledge Graphs](https://mlsanity.com/paper/2203.01520/) (2022)
- [Understanding the Performance of Knowledge Graph Embeddings in Drug Discovery](https://mlsanity.com/paper/2105.10488/) (2021)
- [A Unified Framework for Rank-based Evaluation Metrics for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/2203.07544/) (2022)
- [Sem@K: Is my knowledge graph embedding model semantic-aware?](https://mlsanity.com/paper/2301.05601/) (2023)
- [Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphs](https://mlsanity.com/paper/2111.10625/) (2021)
- [Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval Perspective](https://mlsanity.com/paper/2205.04105/) (2022)
- [Knowledge Graph Embedding Methods for Entity Alignment: An Experimental Review](https://mlsanity.com/paper/2203.09280/) (2022)
- [Rule Induction in Knowledge Graphs Using Linear Programming](https://mlsanity.com/paper/2110.08245/) (2021)
- [Improving Molecule Generation and Drug Discovery with a Knowledge-enhanced Generative Model](https://mlsanity.com/paper/2402.08790/) (2024)
- [DuetGraph: Coarse-to-Fine Knowledge Graph Reasoning with Dual-Pathway Global-Local Fusion](https://mlsanity.com/paper/2507.11229/) (2025)
- [PDPK: A Framework to Synthesise Process Data and Corresponding Procedural Knowledge for Manufacturing](https://mlsanity.com/paper/2308.08371/) (2023)
- [Generalized Rank-based Evaluation for Knowledge Graph Completion: Perspectives, Framework, and Analyses](https://mlsanity.com/paper/2606.08921/) (2026)

## Abstract (arXiv)

In this work, we take a closer look at the evaluation of two families of methods for enriching information from knowledge graphs: Link Prediction and Entity Alignment. In the current experimental setting, multiple different scores are employed to assess different aspects of model performance. We analyze the informativeness of these evaluation measures and identify several shortcomings. In particular, we demonstrate that all existing scores can hardly be used to compare results across different datasets. Moreover, we demonstrate that varying size of the test size automatically has impact on the performance of the same model based on commonly used metrics for the Entity Alignment task. We show that this leads to various problems in the interpretation of results, which may support misleading conclusions. Therefore, we propose adjustments to the evaluation and demonstrate empirically how this supports a fair, comparable, and interpretable assessment of model performance. Our code is available at https://github.com/mberr/rank-based-evaluation.