# On Evaluating Embedding Models for Knowledge Base Completion

arXiv: https://arxiv.org/abs/1810.07180 · HTML: https://mlsanity.com/paper/1810.07180/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2018.

## Built on by (most cited first, 10 of 10)

- [A Unified Framework for Rank-based Evaluation Metrics for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/2203.07544/) (2022)
- [Bilinear Scoring Function Search for Knowledge Graph Learning](https://mlsanity.com/paper/2107.00184/) (2021)
- [Sem@K: Is my knowledge graph embedding model semantic-aware?](https://mlsanity.com/paper/2301.05601/) (2023)
- [Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval Perspective](https://mlsanity.com/paper/2205.04105/) (2022)
- [Ranking vs. Classifying: Measuring Knowledge Base Completion Quality](https://mlsanity.com/paper/2102.06145/) (2021)
- [KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Models](https://mlsanity.com/paper/2208.11024/) (2022)
- [On Large-scale Evaluation of Embedding Models for Knowledge Graph Completion](https://mlsanity.com/paper/2504.08970/) (2025)
- [Pseudo-Riemannian Embedding Models for Multi-Relational Graph Representations](https://mlsanity.com/paper/2212.03720/) (2022)
- [Pharmacology Knowledge Graphs: Do We Need Chemical Structure for Drug Repurposing?](https://mlsanity.com/paper/2603.01537/) (2026)
- [Pattern Over-Generalization of Knowledge Graph Embedding](https://mlsanity.com/paper/2609.03487/) (2026)

## Abstract (arXiv)

Knowledge bases contribute to many web search and mining tasks, yet they are often incomplete. To add missing facts to a given knowledge base, various embedding models have been proposed in the recent literature. Perhaps surprisingly, relatively simple models with limited expressiveness often performed remarkably well under today's most commonly used evaluation protocols. In this paper, we explore whether recent models work well for knowledge base completion and argue that the current evaluation protocols are more suited for question answering rather than knowledge base completion. We show that when focusing on a different prediction task for evaluating knowledge base completion, the performance of current embedding models is unsatisfactory even on datasets previously thought to be too easy. This is especially true when embedding models are compared against a simple rule-based baseline. This work indicates the need for more research into the embedding models and evaluation protocols for knowledge base completion.