# Inductive Relation Prediction from Relational Paths and Context with Hierarchical Transformers

arXiv: https://arxiv.org/abs/2304.00215 · HTML: https://mlsanity.com/paper/2304.00215/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 1 Apr 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Inductive Relation Prediction by Subgraph Reasoning](https://mlsanity.com/paper/1911.06962/) (2019)
- [DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs](https://mlsanity.com/paper/1911.00055/) (2019)
- [NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs](https://mlsanity.com/paper/2106.12144/) (2021)
- [Relational Message Passing for Knowledge Graph Completion](https://mlsanity.com/paper/2002.06757/) (2020)
- [Modeling Relation Paths for Representation Learning of Knowledge Bases](https://mlsanity.com/paper/1506.00379/) (2015)
- [HittER: Hierarchical Transformers for Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.12813/) (2020)
- [Communicative Message Passing for Inductive Relation Reasoning](https://mlsanity.com/paper/2012.08911/) (2020)
- [Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/2103.03642/) (2021)
- [Meta-Knowledge Transfer for Inductive Knowledge Graph Embedding](https://mlsanity.com/paper/2110.14170/) (2021)
- [Learning First-Order Rules with Relational Path Contrast for Inductive Relation Reasoning](https://mlsanity.com/paper/2110.08810/) (2021)

## Built on by (most cited first, 4 of 4)

- [KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning](https://mlsanity.com/paper/2409.12865/) (2024)
- [Random Entity Quantization for Parameter-Efficient Compositional Knowledge Graph Representation](https://mlsanity.com/paper/2310.15797/) (2023)
- [Replacing Paths with Connection-Biased Attention for Knowledge Graph Completion](https://mlsanity.com/paper/2410.00876/) (2024)
- [Beyond Transduction: A Survey on Inductive, Few Shot, and Zero Shot Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/2312.04997/) (2023)

## Abstract (arXiv)

Relation prediction on knowledge graphs (KGs) is a key research topic. Dominant embedding-based methods mainly focus on the transductive setting and lack the inductive ability to generalize to new entities for inference. Existing methods for inductive reasoning mostly mine the connections between entities, i.e., relational paths, without considering the nature of head and tail entities contained in the relational context. This paper proposes a novel method that captures both connections between entities and the intrinsic nature of entities, by simultaneously aggregating RElational Paths and cOntext with a unified hieRarchical Transformer framework, namely REPORT. REPORT relies solely on relation semantics and can naturally generalize to the fully-inductive setting, where KGs for training and inference have no common entities. In the experiments, REPORT performs consistently better than all baselines on almost all the eight version subsets of two fully-inductive datasets. Moreover. REPORT is interpretable by providing each element's contribution to the prediction results.