# Convolutional Hypercomplex Embeddings for Link Prediction arXiv: https://arxiv.org/abs/2106.15230 · HTML: https://mlsanity.com/paper/2106.15230/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Builds on 28 of its 45 references that are in the map. - First posted on arXiv 29 Jun 2021. ## Builds on (most cited first, 28 of 28) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015) - [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014) - [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016) - [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019) - [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017) - [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017) - [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020) - [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020) - [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019) - [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015) - [A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications](https://mlsanity.com/paper/1709.07604/) (2017) - [Low-Dimensional Hyperbolic Knowledge Graph Embeddings](https://mlsanity.com/paper/2005.00545/) (2020) - [Quaternion Knowledge Graph Embeddings](https://mlsanity.com/paper/1904.10281/) (2019) - [Canonical Tensor Decomposition for Knowledge Base Completion](https://mlsanity.com/paper/1806.07297/) (2018) - [A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network](https://mlsanity.com/paper/1712.02121/) (2017) - [Multi-relational Poincar'e Graph Embeddings](https://mlsanity.com/paper/1905.09791/) (2019) - [How to Start Training: The Effect of Initialization and Architecture](https://mlsanity.com/paper/1803.01719/) (2018) - [Knowledge Graph Completion via Complex Tensor Factorization](https://mlsanity.com/paper/1702.06879/) (2017) - [Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters](https://mlsanity.com/paper/2102.08597/) (2021) - [Question Answering with Subgraph Embeddings](https://mlsanity.com/paper/1406.3676/) (2014) - [Hypernetwork Knowledge Graph Embeddings](https://mlsanity.com/paper/1808.07018/) (2018) - [Differentiable Reasoning on Large Knowledge Bases and Natural Language](https://mlsanity.com/paper/1912.10824/) (2019) - [Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/1806.07789/) (2018) - [Convolutional Complex Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.03130/) (2020) - [A shallow neural model for relation prediction](https://mlsanity.com/paper/2101.09090/) (2021) - [Out-of-Vocabulary Entities in Link Prediction](https://mlsanity.com/paper/2105.12524/) (2021) ## Built on by (most cited first, 15 of 15) - [EvoLearner: Learning Description Logics with Evolutionary Algorithms](https://mlsanity.com/paper/2111.04879/) (2021) - [Conformal Credal Self-Supervised Learning](https://mlsanity.com/paper/2205.15239/) (2022) - [LitCQD: Multi-Hop Reasoning in Incomplete Knowledge Graphs with Numeric Literals](https://mlsanity.com/paper/2304.14742/) (2023) - [Adaptive Stochastic Weight Averaging](https://mlsanity.com/paper/2406.19092/) (2024) - [Hardware-agnostic Computation for Large-scale Knowledge Graph Embeddings](https://mlsanity.com/paper/2207.08544/) (2022) - [Kronecker Decomposition for Knowledge Graph Embeddings](https://mlsanity.com/paper/2205.06560/) (2022) - [Learning Permutation-Invariant Embeddings for Description Logic Concepts](https://mlsanity.com/paper/2303.01844/) (2023) - [Universal Knowledge Graph Embeddings](https://mlsanity.com/paper/2310.14899/) (2023) - [Embedding Knowledge Graphs in Degenerate Clifford Algebras](https://mlsanity.com/paper/2402.04870/) (2024) - [HybridFC: A Hybrid Fact-Checking Approach for Knowledge Graphs](https://mlsanity.com/paper/2409.06692/) (2024) - [Embedding Knowledge Graph in Function Spaces](https://mlsanity.com/paper/2409.14857/) (2024) - [Resilience in Knowledge Graph Embeddings](https://mlsanity.com/paper/2410.21163/) (2024) - [Performance Evaluation of Knowledge Graph Embedding Approaches under Non-adversarial Attacks](https://mlsanity.com/paper/2407.06855/) (2024) - [Inference over Unseen Entities, Relations and Literals on Knowledge Graphs](https://mlsanity.com/paper/2410.06742/) (2024) - [Neural Regression with Embeddings for Numerical Attribute Prediction in Knowledge Graphs](https://mlsanity.com/paper/2608.26729/) (2026) ## Abstract (arXiv) Knowledge graph embedding research has mainly focused on the two smallest normed division algebras, ℝ and mathbbC. Recent results suggest that trilinear products of quaternion-valued embeddings can be a more effective means to tackle link prediction. In addition, models based on convolutions on real-valued embeddings often yield state-of-the-art results for link prediction. In this paper, we investigate a composition of convolution operations with hypercomplex multiplications. We propose the four approaches QMult, OMult, ConvQ and ConvO to tackle the link prediction problem. QMult and OMult can be considered as quaternion and octonion extensions of previous state-of-the-art approaches, including DistMult and ComplEx. ConvQ and ConvO build upon QMult and OMult by including convolution operations in a way inspired by the residual learning framework. We evaluated our approaches on seven link prediction datasets including WN18RR, FB15K-237 and YAGO3-10. Experimental results suggest that the benefits of learning hypercomplex-valued vector representations become more apparent as the size and complexity of the knowledge graph grows. ConvO outperforms state-of-the-art approaches on FB15K-237 in MRR, Hit@1 and Hit@3, while QMult, OMult, ConvQ and ConvO outperform state-of-the-approaches on YAGO3-10 in all metrics. Results also suggest that link prediction performances can be further improved via prediction averaging. To foster reproducible research, we provide an open-source implementation of approaches, including training and evaluation scripts as well as pretrained models.