# Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations

arXiv: https://arxiv.org/abs/2110.02834 · HTML: https://mlsanity.com/paper/2110.02834/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 6 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015)
- [Canonical Tensor Decomposition for Knowledge Base Completion](https://mlsanity.com/paper/1806.07297/) (2018)
- [A Re-evaluation of Knowledge Graph Completion Methods](https://mlsanity.com/paper/1911.03903/) (2019)
- [Knowledge Base Completion: Baselines Strike Back](https://mlsanity.com/paper/1705.10744/) (2017)
- [Dynamically Pruned Message Passing Networks for Large-Scale Knowledge Graph Reasoning](https://mlsanity.com/paper/1909.11334/) (2019)
- [Knowledge Base Completion: Baseline strikes back (Again)](https://mlsanity.com/paper/2005.00804/) (2020)

## Built on by (most cited first, 30 of 37)

- [Towards Foundation Models for Knowledge Graph Reasoning](https://mlsanity.com/paper/2310.04562/) (2023)
- [When Do Flat Minima Optimizers Work?](https://mlsanity.com/paper/2202.00661/) (2022)
- [A*Net: A Scalable Path-based Reasoning Approach for Knowledge Graphs](https://mlsanity.com/paper/2206.04798/) (2022)
- [Answering Complex Logical Queries on Knowledge Graphs via Query Computation Tree Optimization](https://mlsanity.com/paper/2212.09567/) (2022)
- [A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning](https://mlsanity.com/paper/2410.12288/) (2024)
- [A Multi-Task Perspective for Link Prediction with New Relation Types and Nodes](https://mlsanity.com/paper/2307.06046/) (2023)
- [TRIX: A More Expressive Model for Zero-shot Domain Transfer in Knowledge Graphs](https://mlsanity.com/paper/2502.19512/) (2025)
- [How to Turn Your Knowledge Graph Embeddings into Generative Models](https://mlsanity.com/paper/2305.15944/) (2023)
- [Multi-task Pre-training Language Model for Semantic Network Completion](https://mlsanity.com/paper/2201.04843/) (2022)
- [Joint Language Semantic and Structure Embedding for Knowledge Graph Completion](https://mlsanity.com/paper/2209.08721/) (2022)
- [ReFactor GNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective](https://mlsanity.com/paper/2207.09980/) (2022)
- [TripleRE: Knowledge Graph Embeddings via Tripled Relation Vectors](https://mlsanity.com/paper/2209.08271/) (2022)
- [Flock: A Knowledge Graph Foundation Model via Learning on Random Walks](https://mlsanity.com/paper/2510.01510/) (2025)
- [AsEP: Benchmarking Deep Learning Methods for Antibody-specific Epitope Prediction](https://mlsanity.com/paper/2407.18184/) (2024)
- [SEMMA: A Semantic Aware Knowledge Graph Foundation Model](https://mlsanity.com/paper/2505.20422/) (2025)
- [TranS: Transition-based Knowledge Graph Embedding with Synthetic Relation Representation](https://mlsanity.com/paper/2204.08401/) (2022)
- [Large Language Model Enhanced Knowledge Representation Learning: A Survey](https://mlsanity.com/paper/2407.00936/) (2024)
- [InterHT: Knowledge Graph Embeddings by Interaction between Head and Tail Entities](https://mlsanity.com/paper/2202.04897/) (2022)
- [How Data Inter-connectivity Shapes LLMs Unlearning: A Structural Unlearning Perspective](https://mlsanity.com/paper/2406.16810/) (2024)
- [Commonsense Knowledge Salience Evaluation with a Benchmark Dataset in E-commerce](https://mlsanity.com/paper/2205.10843/) (2022)
- [Query2Triple: Unified Query Encoding for Answering Diverse Complex Queries over Knowledge Graphs](https://mlsanity.com/paper/2310.11246/) (2023)
- [SnapE -- Training Snapshot Ensembles of Link Prediction Models](https://mlsanity.com/paper/2408.02707/) (2024)
- [Knowledge Graph Embedding with 3D Compound Geometric Transformations](https://mlsanity.com/paper/2304.00378/) (2023)
- [Knowledge Graph Completion with Mixed Geometry Tensor Factorization](https://mlsanity.com/paper/2504.02589/) (2025)
- [Relation-aware Ensemble Learning for Knowledge Graph Embedding](https://mlsanity.com/paper/2310.08917/) (2023)
- [Self-Distillation with Meta Learning for Knowledge Graph Completion](https://mlsanity.com/paper/2305.12209/) (2023)
- [Towards Better Benchmark Datasets for Inductive Knowledge Graph Completion](https://mlsanity.com/paper/2406.11898/) (2024)
- [Complex Logical Query Answering by Calibrating Knowledge Graph Completion Models](https://mlsanity.com/paper/2410.07165/) (2024)
- [Efficient and Scalable Neural Symbolic Search for Knowledge Graph Complex Query Answering](https://mlsanity.com/paper/2505.08155/) (2025)
- [Geometric Structural Knowledge Graph Foundation Model](https://mlsanity.com/paper/2512.22931/) (2025)

## Abstract (arXiv)

Learning good representations on multi-relational graphs is essential to knowledge base completion (KBC). In this paper, we propose a new self-supervised training objective for multi-relational graph representation learning, via simply incorporating relation prediction into the commonly used 1vsAll objective. The new training objective contains not only terms for predicting the subject and object of a given triple, but also a term for predicting the relation type. We analyse how this new objective impacts multi-relational learning in KBC: experiments on a variety of datasets and models show that relation prediction can significantly improve entity ranking, the most widely used evaluation task for KBC, yielding a 6.1% increase in MRR and 9.9% increase in Hits@1 on FB15k-237 as well as a 3.1% increase in MRR and 3.4% in Hits@1 on Aristo-v4. Moreover, we observe that the proposed objective is especially effective on highly multi-relational datasets, i.e. datasets with a large number of predicates, and generates better representations when larger embedding sizes are used.