# A shallow neural model for relation prediction

arXiv: https://arxiv.org/abs/2101.09090 · HTML: https://mlsanity.com/paper/2101.09090/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 22 Jan 2021.

## Builds on (most cited first, 19 of 19)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [The Hardware Lottery](https://mlsanity.com/paper/2009.06489/) (2020)
- [Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review](https://mlsanity.com/paper/1611.00740/) (2016)
- [Knowledge Graph Completion via Complex Tensor Factorization](https://mlsanity.com/paper/1702.06879/) (2017)
- [Type-Constrained Representation Learning in Knowledge Graphs](https://mlsanity.com/paper/1508.02593/) (2015)
- [ProjE: Embedding Projection for Knowledge Graph Completion](https://mlsanity.com/paper/1611.05425/) (2016)
- [Convolutional Complex Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.03130/) (2020)
- [SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions](https://mlsanity.com/paper/1604.04835/) (2016)
- [Neural Networks Should Be Wide Enough to Learn Disconnected Decision Regions](https://mlsanity.com/paper/1803.00094/) (2018)

## Built on by (most cited first, 6 of 6)

- [Convolutional Hypercomplex Embeddings for Link Prediction](https://mlsanity.com/paper/2106.15230/) (2021)
- [Out-of-Vocabulary Entities in Link Prediction](https://mlsanity.com/paper/2105.12524/) (2021)
- [DRILL-- Deep Reinforcement Learning for Refinement Operators in ALC](https://mlsanity.com/paper/2106.15373/) (2021)
- [Hardware-agnostic Computation for Large-scale Knowledge Graph Embeddings](https://mlsanity.com/paper/2207.08544/) (2022)
- [Knowledge Graph Completion using Structural and Textual Embeddings](https://mlsanity.com/paper/2404.16206/) (2024)
- [Relations Prediction for Knowledge Graph Completion using Large Language Models](https://mlsanity.com/paper/2405.02738/) (2024)

## Abstract (arXiv)

Knowledge graph completion refers to predicting missing triples. Most approaches achieve this goal by predicting entities, given an entity and a relation. We predict missing triples via the relation prediction. To this end, we frame the relation prediction problem as a multi-label classification problem and propose a shallow neural model (SHALLOM) that accurately infers missing relations from entities. SHALLOM is analogous to C-BOW as both approaches predict a central token (p) given surrounding tokens ((s,o)). Our experiments indicate that SHALLOM outperforms state-of-the-art approaches on the FB15K-237 and WN18RR with margins of up to 3% and 8% (absolute), respectively, while requiring a maximum training time of 8 minutes on these datasets. We ensure the reproducibility of our results by providing an open-source implementation including training and evaluation scripts at urlhttps://github.com/dice-group/Shallom.