# Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completion

arXiv: https://arxiv.org/abs/2108.01387 · HTML: https://mlsanity.com/paper/2108.01387/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 3 Aug 2021.

## Builds on (most cited first, 22 of 22)

- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning](https://mlsanity.com/paper/1707.06690/) (2017)
- [Holographic Embeddings of Knowledge Graphs](https://mlsanity.com/paper/1510.04935/) (2015)
- [Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning](https://mlsanity.com/paper/1711.05851/) (2017)
- [BoxE: A Box Embedding Model for Knowledge Base Completion](https://mlsanity.com/paper/2007.06267/) (2020)
- [Multi-Hop Knowledge Graph Reasoning with Reward Shaping](https://mlsanity.com/paper/1808.10568/) (2018)
- [CoDEx: A Comprehensive Knowledge Graph Completion Benchmark](https://mlsanity.com/paper/2009.07810/) (2020)
- [Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences](https://mlsanity.com/paper/1902.06236/) (2019)
- [Multi-Channel Graph Neural Network for Entity Alignment](https://mlsanity.com/paper/1908.09898/) (2019)
- [Open-World Knowledge Graph Completion](https://mlsanity.com/paper/1711.03438/) (2017)
- [A Logic-Driven Framework for Consistency of Neural Models](https://mlsanity.com/paper/1909.00126/) (2019)
- [Few-Shot Knowledge Graph Completion](https://mlsanity.com/paper/1911.11298/) (2019)
- [Knowledge Graph Embedding with Iterative Guidance from Soft Rules](https://mlsanity.com/paper/1711.11231/) (2017)
- [Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen](https://mlsanity.com/paper/2005.00701/) (2020)
- [Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study](https://mlsanity.com/paper/2003.08001/) (2020)
- [Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1708.06816/) (2017)
- [Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction](https://mlsanity.com/paper/2011.13574/) (2020)
- [How Knowledge Graph and Attention Help? A Quantitative Analysis into Bag-level Relation Extraction](https://mlsanity.com/paper/2107.12064/) (2021)

## Built on by (most cited first, 9 of 9)

- [SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models](https://mlsanity.com/paper/2203.02167/) (2022)
- [Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment](https://mlsanity.com/paper/2203.14987/) (2022)
- [Rethinking Knowledge Graph Evaluation Under the Open-World Assumption](https://mlsanity.com/paper/2209.08858/) (2022)
- [SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2210.04870/) (2022)
- [ForecastTKGQuestions: A Benchmark for Temporal Question Answering and Forecasting over Temporal Knowledge Graphs](https://mlsanity.com/paper/2208.06501/) (2022)
- [KERMIT: Knowledge Graph Completion of Enhanced Relation Modeling with Inverse Transformation](https://mlsanity.com/paper/2309.14770/) (2023)
- [KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction](https://mlsanity.com/paper/2404.15923/) (2024)
- [Revisiting Inferential Benchmarks for Knowledge Graph Completion](https://mlsanity.com/paper/2306.04814/) (2023)
- [Counterfactual Reasoning with Knowledge Graph Embeddings](https://mlsanity.com/paper/2403.06936/) (2024)

## Abstract (arXiv)

We present InferWiki, a Knowledge Graph Completion (KGC) dataset that improves upon existing benchmarks in inferential ability, assumptions, and patterns. First, each testing sample is predictable with supportive data in the training set. To ensure it, we propose to utilize rule-guided train/test generation, instead of conventional random split. Second, InferWiki initiates the evaluation following the open-world assumption and improves the inferential difficulty of the closed-world assumption, by providing manually annotated negative and unknown triples. Third, we include various inference patterns (e.g., reasoning path length and types) for comprehensive evaluation. In experiments, we curate two settings of InferWiki varying in sizes and structures, and apply the construction process on CoDEx as comparative datasets. The results and empirical analyses demonstrate the necessity and high-quality of InferWiki. Nevertheless, the performance gap among various inferential assumptions and patterns presents the difficulty and inspires future research direction. Our datasets can be found in https://github.com/TaoMiner/inferwiki