# MLMLM: Link Prediction with Mean Likelihood Masked Language Model

arXiv: https://arxiv.org/abs/2009.07058 · HTML: https://mlsanity.com/paper/2009.07058/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2020.

## Built on by (most cited first, 22 of 22)

- [Sequence-to-Sequence Knowledge Graph Completion and Question Answering](https://mlsanity.com/paper/2203.10321/) (2022)
- [NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs](https://mlsanity.com/paper/2106.12144/) (2021)
- [Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality](https://mlsanity.com/paper/2211.00768/) (2022)
- [Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Prompting](https://mlsanity.com/paper/2307.01709/) (2023)
- [Improving Inductive Link Prediction Using Hyper-Relational Facts](https://mlsanity.com/paper/2107.04894/) (2021)
- [Multi-task Pre-training Language Model for Semantic Network Completion](https://mlsanity.com/paper/2201.04843/) (2022)
- [I Know What You Do Not Know: Knowledge Graph Embedding via Co-distillation Learning](https://mlsanity.com/paper/2208.09828/) (2022)
- [RAILD: Towards Leveraging Relation Features for Inductive Link Prediction In Knowledge Graphs](https://mlsanity.com/paper/2211.11407/) (2022)
- [Friendly Neighbors: Contextualized Sequence-to-Sequence Link Prediction](https://mlsanity.com/paper/2305.13059/) (2023)
- [Logical Entity Representation in Knowledge-Graphs for Differentiable Rule Learning](https://mlsanity.com/paper/2305.12738/) (2023)
- [Large Language Model Enhanced Knowledge Representation Learning: A Survey](https://mlsanity.com/paper/2407.00936/) (2024)
- [Bridging the Language Gap: Knowledge Injected Multilingual Question Answering](https://mlsanity.com/paper/2304.03159/) (2023)
- [KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Models](https://mlsanity.com/paper/2208.11024/) (2022)
- [Knowledge Graph Error Detection with Contrastive Confidence Adaption](https://mlsanity.com/paper/2312.12108/) (2023)
- [Integrating Knowledge Graph embedding and pretrained Language Models in Hypercomplex Spaces](https://mlsanity.com/paper/2208.02743/) (2022)
- [Pre-training Transformers for Knowledge Graph Completion](https://mlsanity.com/paper/2303.15682/) (2023)
- [Beyond Transduction: A Survey on Inductive, Few Shot, and Zero Shot Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/2312.04997/) (2023)
- [Large Language Model-Enhanced Symbolic Reasoning for Knowledge Base Completion](https://mlsanity.com/paper/2501.01246/) (2025)
- [Enhancing Knowledge Graph Completion with Entity Neighborhood and Relation Context](https://mlsanity.com/paper/2503.23205/) (2025)
- [KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Models](https://mlsanity.com/paper/2506.03576/) (2025)
- [Pre-training and Diagnosing Knowledge Base Completion Models](https://mlsanity.com/paper/2401.15439/) (2024)
- [KG-TRICK: Unifying Textual and Relational Information Completion of Knowledge for Multilingual Knowledge Graphs](https://mlsanity.com/paper/2501.03560/) (2025)

## Abstract (arXiv)

Knowledge Bases (KBs) are easy to query, verifiable, and interpretable. They however scale with man-hours and high-quality data. Masked Language Models (MLMs), such as BERT, scale with computing power as well as unstructured raw text data. The knowledge contained within those models is however not directly interpretable. We propose to perform link prediction with MLMs to address both the KBs scalability issues and the MLMs interpretability issues. To do that we introduce MLMLM, Mean Likelihood Masked Language Model, an approach comparing the mean likelihood of generating the different entities to perform link prediction in a tractable manner. We obtain State of the Art (SotA) results on the WN18RR dataset and the best non-entity-embedding based results on the FB15k-237 dataset. We also obtain convincing results on link prediction on previously unseen entities, making MLMLM a suitable approach to introducing new entities to a KB.