# Type-aware Embeddings for Multi-Hop Reasoning over Knowledge Graphs

arXiv: https://arxiv.org/abs/2205.00782 · HTML: https://mlsanity.com/paper/2205.00782/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 2 May 2022.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 22 of 22)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning](https://mlsanity.com/paper/1707.06690/) (2017)
- [Highway Networks](https://mlsanity.com/paper/1505.00387/) (2015)
- [Inductive Relation Prediction by Subgraph Reasoning](https://mlsanity.com/paper/1911.06962/) (2019)
- [Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings](https://mlsanity.com/paper/2002.05969/) (2020)
- [Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs](https://mlsanity.com/paper/2010.11465/) (2020)
- [Embedding Logical Queries on Knowledge Graphs](https://mlsanity.com/paper/1806.01445/) (2018)
- [Complex Query Answering with Neural Link Predictors](https://mlsanity.com/paper/2011.03459/) (2020)
- [ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2110.13715/) (2021)
- [Inductive Entity Representations from Text via Link Prediction](https://mlsanity.com/paper/2010.03496/) (2020)
- [Relational Message Passing for Knowledge Graph Completion](https://mlsanity.com/paper/2002.06757/) (2020)
- [Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/2103.03642/) (2021)
- [Self-Supervised Hyperboloid Representations from Logical Queries over Knowledge Graphs](https://mlsanity.com/paper/2012.13023/) (2020)
- [Probabilistic Entity Representation Model for Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2110.13522/) (2021)
- [Dual Co-Matching Network for Multi-choice Reading Comprehension](https://mlsanity.com/paper/1901.09381/) (2019)
- [Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention](https://mlsanity.com/paper/1605.09090/) (2016)
- [Connecting Embeddings for Knowledge Graph Entity Typing](https://mlsanity.com/paper/2007.10873/) (2020)
- [Logic Embeddings for Complex Query Answering](https://mlsanity.com/paper/2103.00418/) (2021)
- [Context-aware Entity Typing in Knowledge Graphs](https://mlsanity.com/paper/2109.07990/) (2021)
- [AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding](https://mlsanity.com/paper/2009.12030/) (2020)

## Built on by (most cited first, 13 of 13)

- [A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal](https://mlsanity.com/paper/2212.05767/) (2022)
- [Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databases](https://mlsanity.com/paper/2303.14617/) (2023)
- [Transformer-based Entity Typing in Knowledge Graphs](https://mlsanity.com/paper/2210.11151/) (2022)
- [EFOk-CQA: Towards Knowledge Graph Complex Query Answering beyond Set Operation](https://mlsanity.com/paper/2307.13701/) (2023)
- [Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding](https://mlsanity.com/paper/2302.01849/) (2023)
- [HyperFormer: Enhancing Entity and Relation Interaction for Hyper-Relational Knowledge Graph Completion](https://mlsanity.com/paper/2308.06512/) (2023)
- [Inductive Knowledge Graph Completion with GNNs and Rules: An Analysis](https://mlsanity.com/paper/2308.07942/) (2023)
- [Query Structure Modeling for Inductive Logical Reasoning Over Knowledge Graphs](https://mlsanity.com/paper/2305.13585/) (2023)
- [HyperMono: A Monotonicity-aware Approach to Hyper-Relational Knowledge Representation](https://mlsanity.com/paper/2404.09848/) (2024)
- [Leveraging Intra-modal and Inter-modal Interaction for Multi-Modal Entity Alignment](https://mlsanity.com/paper/2404.17590/) (2024)
- [Type-based Neural Link Prediction Adapter for Complex Query Answering](https://mlsanity.com/paper/2401.16045/) (2024)
- [Prompt-fused framework for Inductive Logical Query Answering](https://mlsanity.com/paper/2403.12646/) (2024)
- [COTET: Cross-view Optimal Transport for Knowledge Graph Entity Typing](https://mlsanity.com/paper/2405.13602/) (2024)

## Abstract (arXiv)

Multi-hop reasoning over real-life knowledge graphs (KGs) is a highly challenging problem as traditional subgraph matching methods are not capable to deal with noise and missing information. To address this problem, it has been recently introduced a promising approach based on jointly embedding logical queries and KGs into a low-dimensional space to identify answer entities. However, existing proposals ignore critical semantic knowledge inherently available in KGs, such as type information. To leverage type information, we propose a novel TypE-aware Message Passing (TEMP) model, which enhances the entity and relation representations in queries, and simultaneously improves generalization, deductive and inductive reasoning. Remarkably, TEMP is a plug-and-play model that can be easily incorporated into existing embedding-based models to improve their performance. Extensive experiments on three real-world datasets demonstrate TEMP's effectiveness.