# Transformers for Complex Query Answering over Knowledge Hypergraphs

arXiv: https://arxiv.org/abs/2504.16537 · HTML: https://mlsanity.com/paper/2504.16537/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 23 Apr 2025.

## Builds on (most cited first, 29 of 29)

- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings](https://mlsanity.com/paper/2002.05969/) (2020)
- [Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs](https://mlsanity.com/paper/2010.11465/) (2020)
- [Embedding Logical Queries on Knowledge Graphs](https://mlsanity.com/paper/1806.01445/) (2018)
- [BoxE: A Box Embedding Model for Knowledge Base Completion](https://mlsanity.com/paper/2007.06267/) (2020)
- [Complex Query Answering with Neural Link Predictors](https://mlsanity.com/paper/2011.03459/) (2020)
- [Message Passing for Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2009.10847/) (2020)
- [ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2110.13715/) (2021)
- [On the representation and embedding of knowledge bases beyond binary relations](https://mlsanity.com/paper/1604.08642/) (2016)
- [Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases](https://mlsanity.com/paper/2007.03988/) (2020)
- [Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databases](https://mlsanity.com/paper/2303.14617/) (2023)
- [Link Prediction on N-ary Relational Facts: A Graph-based Approach](https://mlsanity.com/paper/2105.08476/) (2021)
- [Answering Complex Logical Queries on Knowledge Graphs via Query Computation Tree Optimization](https://mlsanity.com/paper/2212.09567/) (2022)
- [Logical Message Passing Networks with One-hop Inference on Atomic Formulas](https://mlsanity.com/paper/2301.08859/) (2023)
- [Benchmarking the Combinatorial Generalizability of Complex Query Answering on Knowledge Graphs](https://mlsanity.com/paper/2109.08925/) (2021)
- [Neural-Symbolic Entangled Framework for Complex Query Answering](https://mlsanity.com/paper/2209.08779/) (2022)
- [Sequential Query Encoding For Complex Query Answering on Knowledge Graphs](https://mlsanity.com/paper/2302.13114/) (2023)
- [Knowledge Hypergraph Embedding Meets Relational Algebra](https://mlsanity.com/paper/2102.09557/) (2021)
- [NQE: N-ary Query Embedding for Complex Query Answering over Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2211.13469/) (2022)
- [Understanding Inter-Session Intentions via Complex Logical Reasoning](https://mlsanity.com/paper/2312.13866/) (2023)
- [Enhancing Transformers for Generalizable First-Order Logical Entailment](https://mlsanity.com/paper/2501.00759/) (2025)
- [Generalizing Hyperedge Expansion for Hyper-relational Knowledge Graph Modeling](https://mlsanity.com/paper/2411.06191/) (2024)
- [Pathformer: Recursive Path Query Encoding for Complex Logical Query Answering](https://mlsanity.com/paper/2406.14880/) (2024)

## Built on by (most cited first, 2 of 2)

- [Efficient and Scalable Neural Symbolic Search for Knowledge Graph Complex Query Answering](https://mlsanity.com/paper/2505.08155/) (2025)
- [Two-dimensional Taxonomy for N-ary Knowledge Representation Learning Methods](https://mlsanity.com/paper/2506.05626/) (2025)

## Abstract (arXiv)

Complex Query Answering (CQA) has been extensively studied in recent years. In order to model data that is closer to real-world distribution, knowledge graphs with different modalities have been introduced. Triple KGs, as the classic KGs composed of entities and relations of arity 2, have limited representation of real-world facts. Real-world data is more sophisticated. While hyper-relational graphs have been introduced, there are limitations in representing relationships of varying arity that contain entities with equal contributions. To address this gap, we sampled new CQA datasets: JF17k-HCQA and M-FB15k-HCQA. Each dataset contains various query types that include logical operations such as projection, negation, conjunction, and disjunction. In order to answer knowledge hypergraph (KHG) existential first-order queries, we propose a two-stage transformer model, the Logical Knowledge Hypergraph Transformer (LKHGT), which consists of a Projection Encoder for atomic projection and a Logical Encoder for complex logical operations. Both encoders are equipped with Type Aware Bias (TAB) for capturing token interactions. Experimental results on CQA datasets show that LKHGT is a state-of-the-art CQA method over KHG and is able to generalize to out-of-distribution query types.