# Knowledge Base Completion Meets Transfer Learning

arXiv: https://arxiv.org/abs/2108.13073 · HTML: https://mlsanity.com/paper/2108.13073/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [Canonical Tensor Decomposition for Knowledge Base Completion](https://mlsanity.com/paper/1806.07297/) (2018)
- [PyTorch-BigGraph: A Large-scale Graph Embedding System](https://mlsanity.com/paper/1903.12287/) (2019)
- [BoxE: A Box Embedding Model for Knowledge Base Completion](https://mlsanity.com/paper/2007.06267/) (2020)
- [Knowledge Graph Completion via Complex Tensor Factorization](https://mlsanity.com/paper/1702.06879/) (2017)
- [A Survey on Open Information Extraction](https://mlsanity.com/paper/1806.05599/) (2018)
- [CESI: Canonicalizing Open Knowledge Bases using Embeddings and Side Information](https://mlsanity.com/paper/1902.00172/) (2019)
- [Duality-Induced Regularizer for Tensor Factorization Based Knowledge Graph Completion](https://mlsanity.com/paper/2011.05816/) (2020)
- [5* Knowledge Graph Embeddings with Projective Transformations](https://mlsanity.com/paper/2006.04986/) (2020)
- [OKGIT: Open Knowledge Graph Link Prediction with Implicit Types](https://mlsanity.com/paper/2106.12806/) (2021)

## Built on by (most cited first, 4 of 4)

- [mOKB6: A Multilingual Open Knowledge Base Completion Benchmark](https://mlsanity.com/paper/2211.06959/) (2022)
- [CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2205.08012/) (2022)
- [Pre-training Transformers for Knowledge Graph Completion](https://mlsanity.com/paper/2303.15682/) (2023)
- [Pre-training and Diagnosing Knowledge Base Completion Models](https://mlsanity.com/paper/2401.15439/) (2024)

## Abstract (arXiv)

The aim of knowledge base completion is to predict unseen facts from existing facts in knowledge bases. In this work, we introduce the first approach for transfer of knowledge from one collection of facts to another without the need for entity or relation matching. The method works for both canonicalized knowledge bases and uncanonicalized or open knowledge bases, i.e., knowledge bases where more than one copy of a real-world entity or relation may exist. Such knowledge bases are a natural output of automated information extraction tools that extract structured data from unstructured text. Our main contribution is a method that can make use of a large-scale pre-training on facts, collected from unstructured text, to improve predictions on structured data from a specific domain. The introduced method is the most impactful on small datasets such as ReVerb20K, where we obtained 6% absolute increase of mean reciprocal rank and 65% relative decrease of mean rank over the previously best method, despite not relying on large pre-trained models like BERT.