# RelWalk A Latent Variable Model Approach to Knowledge Graph Embedding

arXiv: https://arxiv.org/abs/2101.10070 · HTML: https://mlsanity.com/paper/2101.10070/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 25 Jan 2021.

## Builds on (most cited first, 18 of 18)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015)
- [Canonical Tensor Decomposition for Knowledge Base Completion](https://mlsanity.com/paper/1806.07297/) (2018)
- [KBGAN: Adversarial Learning for Knowledge Graph Embeddings](https://mlsanity.com/paper/1711.04071/) (2017)
- [A Semantic Matching Energy Function for Learning with Multi-relational Data](https://mlsanity.com/paper/1301.3485/) (2013)
- [Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding](https://mlsanity.com/paper/1911.04910/) (2019)
- [Improving Knowledge Graph Embedding Using Simple Constraints](https://mlsanity.com/paper/1805.02408/) (2018)
- [TransG : A Generative Mixture Model for Knowledge Graph Embedding](https://mlsanity.com/paper/1509.05488/) (2015)
- [STransE: a novel embedding model of entities and relationships in knowledge bases](https://mlsanity.com/paper/1606.08140/) (2016)
- [A survey of embedding models of entities and relationships for knowledge graph completion](https://mlsanity.com/paper/1703.08098/) (2017)
- [Using k-way Co-occurrences for Learning Word Embeddings](https://mlsanity.com/paper/1709.01199/) (2017)
- [Rotate King to get Queen: Word Relationships as Orthogonal Transformations in Embedding Space](https://mlsanity.com/paper/1909.00504/) (2019)

## Built on by (most cited first, 1 of 1)

- [Learning to Borrow -- Relation Representation for Without-Mention Entity-Pairs for Knowledge Graph Completion](https://mlsanity.com/paper/2204.13097/) (2022)

## Abstract (arXiv)

Embedding entities and relations of a knowledge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities. Although progresses have been achieved, existing methods are heuristically motivated and theoretical understanding of such embeddings is comparatively underdeveloped. This paper extends the random walk model (Arora et al., 2016a) of word embeddings to Knowledge Graph Embeddings (KGEs) to derive a scoring function that evaluates the strength of a relation R between two entities h (head) and t (tail). Moreover, we show that marginal loss minimisation, a popular objective used in much prior work in KGE, follows naturally from the log-likelihood ratio maximisation under the probabilities estimated from the KGEs according to our theoretical relationship. We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph. Using the derived objective, accurate KGEs are learnt from FB15K237 and WN18RR benchmark datasets, providing empirical evidence in support of the theory.