# KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Models

arXiv: https://arxiv.org/abs/2208.11024 · HTML: https://mlsanity.com/paper/2208.11024/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 61 references that are in the map.
- First posted on arXiv 23 Aug 2022.

## Builds on (most cited first, 29 of 29)

- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [What Will it Take to Fix Benchmarking in Natural Language Understanding?](https://mlsanity.com/paper/2104.02145/) (2021)
- [Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction](https://mlsanity.com/paper/1911.09419/) (2019)
- [CoDEx: A Comprehensive Knowledge Graph Completion Benchmark](https://mlsanity.com/paper/2009.07810/) (2020)
- [PyKEEN 1.0: A Python Library for Training and Evaluating Knowledge Graph Embeddings](https://mlsanity.com/paper/2007.14175/) (2020)
- [Sequence-to-Sequence Knowledge Graph Completion and Question Answering](https://mlsanity.com/paper/2203.10321/) (2022)
- [A Re-evaluation of Knowledge Graph Completion Methods](https://mlsanity.com/paper/1911.03903/) (2019)
- [Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modeling](https://mlsanity.com/paper/1906.07241/) (2019)
- [Domain-specific Knowledge Graphs: A survey](https://mlsanity.com/paper/2011.00235/) (2020)
- [Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases](https://mlsanity.com/paper/2009.11564/) (2020)
- [ExplainaBoard: An Explainable Leaderboard for NLP](https://mlsanity.com/paper/2104.06387/) (2021)
- [Knowledge Base Completion: Baselines Strike Back](https://mlsanity.com/paper/1705.10744/) (2017)
- [MLMLM: Link Prediction with Mean Likelihood Masked Language Model](https://mlsanity.com/paper/2009.07058/) (2020)
- [KGTK: A Toolkit for Large Knowledge Graph Manipulation and Analysis](https://mlsanity.com/paper/2006.00088/) (2020)
- [Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks](https://mlsanity.com/paper/2204.01906/) (2022)
- [Knowledge Graph Embeddings and Explainable AI](https://mlsanity.com/paper/2004.14843/) (2020)
- [LineaRE: Simple but Powerful Knowledge Graph Embedding for Link Prediction](https://mlsanity.com/paper/2004.10037/) (2020)
- [BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation](https://mlsanity.com/paper/2109.06850/) (2021)
- [On Evaluating Embedding Models for Knowledge Base Completion](https://mlsanity.com/paper/1810.07180/) (2018)
- [DataLab: A Platform for Data Analysis and Intervention](https://mlsanity.com/paper/2202.12875/) (2022)
- [AnnIE: An Annotation Platform for Constructing Complete Open Information Extraction Benchmark](https://mlsanity.com/paper/2109.07464/) (2021)
- [Personalized Benchmarking with the Ludwig Benchmarking Toolkit](https://mlsanity.com/paper/2111.04260/) (2021)
- [Fast Few-shot Debugging for NLU Test Suites](https://mlsanity.com/paper/2204.06555/) (2022)

## Built on by (most cited first, 4 of 4)

- [Linking Surface Facts to Large-Scale Knowledge Graphs](https://mlsanity.com/paper/2310.14909/) (2023)
- [Identifying and Consolidating Knowledge Engineering Requirements](https://mlsanity.com/paper/2306.15124/) (2023)
- [Knowledge Graphs are not Created Equal: Exploring the Properties and Structure of Real KGs](https://mlsanity.com/paper/2311.06414/) (2023)
- [TextMineX: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Action](https://mlsanity.com/paper/2509.15098/) (2025)

## Abstract (arXiv)

Knowledge Graphs (KGs) store information in the form of (head, predicate, tail)-triples. To augment KGs with new knowledge, researchers proposed models for KG Completion (KGC) tasks such as link prediction; i.e., answering (h; p; ?) or (?; p; t) queries. Such models are usually evaluated with averaged metrics on a held-out test set. While useful for tracking progress, averaged single-score metrics cannot reveal what exactly a model has learned -- or failed to learn. To address this issue, we propose KGxBoard: an interactive framework for performing fine-grained evaluation on meaningful subsets of the data, each of which tests individual and interpretable capabilities of a KGC model. In our experiments, we highlight the findings that we discovered with the use of KGxBoard, which would have been impossible to detect with standard averaged single-score metrics.