# A Contextualized BERT model for Knowledge Graph Completion

arXiv: https://arxiv.org/abs/2412.11016 · HTML: https://mlsanity.com/paper/2412.11016/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 15 Dec 2024.

## Builds on (most cited first, 12 of 12)

- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction](https://mlsanity.com/paper/1911.09419/) (2019)
- [SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models](https://mlsanity.com/paper/2203.02167/) (2022)
- [HittER: Hierarchical Transformers for Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.12813/) (2020)
- [KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion](https://mlsanity.com/paper/2402.02389/) (2024)
- [Contextualization Distillation from Large Language Model for Knowledge Graph Completion](https://mlsanity.com/paper/2402.01729/) (2024)
- [NNKGC: Improving Knowledge Graph Completion with Node Neighborhoods](https://mlsanity.com/paper/2302.06132/) (2023)
- [A Relational Tucker Decomposition for Multi-Relational Link Prediction](https://mlsanity.com/paper/1902.00898/) (2019)

## Built on by (most cited first, 5 of 5)

- [MuCoS: Efficient Drug Target Discovery via Multi Context Aware Sampling in Knowledge Graphs](https://mlsanity.com/paper/2503.08075/) (2025)
- [MuCoS: Efficient Drug-Target Prediction through Multi-Context-Aware Sampling](https://mlsanity.com/paper/2502.17784/) (2025)
- [Evaluating Knowledge Graph Complexity via Semantic, Spectral, and Structural Metrics for Link Prediction](https://mlsanity.com/paper/2508.15291/) (2025)
- [Evaluating Cumulative Spectral Gradient as a Complexity Measure](https://mlsanity.com/paper/2509.02399/) (2025)
- [Towards Explainable Job Title Matching: Leveraging Semantic Textual Relatedness and Knowledge Graphs](https://mlsanity.com/paper/2509.09522/) (2025)

## Abstract (arXiv)

Knowledge graphs (KGs) are valuable for representing structured, interconnected information across domains, enabling tasks like semantic search, recommendation systems and inference. A pertinent challenge with KGs, however, is that many entities (i.e., heads, tails) or relationships are unknown. Knowledge Graph Completion (KGC) addresses this by predicting these missing nodes or links, enhancing the graph's informational depth and utility. Traditional methods like TransE and ComplEx predict tail entities but struggle with unseen entities. Textual-based models leverage additional semantics but come with high computational costs, semantic inconsistencies, and data imbalance issues. Recent LLM-based models show improvement but overlook contextual information and rely heavily on entity descriptions. In this study, we introduce a contextualized BERT model for KGC that overcomes these limitations by utilizing the contextual information from neighbouring entities and relationships to predict tail entities. Our model eliminates the need for entity descriptions and negative triplet sampling, reducing computational demands while improving performance. Our model outperforms state-of-the-art methods on standard datasets, improving Hit@1 by 5.3% and 4.88% on FB15k-237 and WN18RR respectively, setting a new benchmark in KGC.