# HyperFormer: Enhancing Entity and Relation Interaction for Hyper-Relational Knowledge Graph Completion arXiv: https://arxiv.org/abs/2308.06512 ยท HTML: https://mlsanity.com/paper/2308.06512/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 48 references that are in the map. - First posted on arXiv 12 Aug 2023. ## Builds on (most cited first, 30 of 30) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016) - [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017) - [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014) - [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016) - [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019) - [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019) - [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019) - [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019) - [Low-Dimensional Hyperbolic Knowledge Graph Embeddings](https://mlsanity.com/paper/2005.00545/) (2020) - [Quaternion Knowledge Graph Embeddings](https://mlsanity.com/paper/1904.10281/) (2019) - [Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction](https://mlsanity.com/paper/1911.09419/) (2019) - [Multi-relational Poincar'e Graph Embeddings](https://mlsanity.com/paper/1905.09791/) (2019) - [Message Passing for Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2009.10847/) (2020) - [InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions](https://mlsanity.com/paper/1911.00219/) (2019) - [On the representation and embedding of knowledge bases beyond binary relations](https://mlsanity.com/paper/1604.08642/) (2016) - [HittER: Hierarchical Transformers for Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.12813/) (2020) - [OntoZSL: Ontology-enhanced Zero-shot Learning](https://mlsanity.com/paper/2102.07339/) (2021) - [Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries](https://mlsanity.com/paper/2208.07638/) (2022) - [Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases](https://mlsanity.com/paper/2007.03988/) (2020) - [Role-Aware Modeling for N-ary Relational Knowledge Bases](https://mlsanity.com/paper/2104.09780/) (2021) - [Link Prediction on N-ary Relational Facts: A Graph-based Approach](https://mlsanity.com/paper/2105.08476/) (2021) - [Improving Hyper-Relational Knowledge Graph Completion](https://mlsanity.com/paper/2104.08167/) (2021) - [I Know What You Do Not Know: Knowledge Graph Embedding via Co-distillation Learning](https://mlsanity.com/paper/2208.09828/) (2022) - [Type-aware Embeddings for Multi-Hop Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2205.00782/) (2022) - [Transformer-based Entity Typing in Knowledge Graphs](https://mlsanity.com/paper/2210.11151/) (2022) - [Personalized News Recommendation with Knowledge-aware Interactive Matching](https://mlsanity.com/paper/2104.10083/) (2021) - [HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level](https://mlsanity.com/paper/2305.06588/) (2023) - [Learning Representations for Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2208.14322/) (2022) ## Built on by (most cited first, 5 of 5) - [Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts](https://mlsanity.com/paper/2405.06524/) (2024) - [Inductive Link Prediction on N-ary Relational Facts via Semantic Hypergraph Reasoning](https://mlsanity.com/paper/2503.20676/) (2025) - [Skeleton-Based Transformer for Classification of Errors and Better Feedback in Low Back Pain Physical Rehabilitation Exercises](https://mlsanity.com/paper/2504.13866/) (2025) - [Understanding the Embedding Models on Hyper-relational Knowledge Graph](https://mlsanity.com/paper/2508.03280/) (2025) - [Generative Representation Learning on Hyper-relational Knowledge Graphs via Masked Discrete Diffusion](https://mlsanity.com/paper/2605.24064/) (2026) ## Abstract (arXiv) Hyper-relational knowledge graphs (HKGs) extend standard knowledge graphs by associating attribute-value qualifiers to triples, which effectively represent additional fine-grained information about its associated triple. Hyper-relational knowledge graph completion (HKGC) aims at inferring unknown triples while considering its qualifiers. Most existing approaches to HKGC exploit a global-level graph structure to encode hyper-relational knowledge into the graph convolution message passing process. However, the addition of multi-hop information might bring noise into the triple prediction process. To address this problem, we propose HyperFormer, a model that considers local-level sequential information, which encodes the content of the entities, relations and qualifiers of a triple. More precisely, HyperFormer is composed of three different modules: an entity neighbor aggregator module allowing to integrate the information of the neighbors of an entity to capture different perspectives of it; a relation qualifier aggregator module to integrate hyper-relational knowledge into the corresponding relation to refine the representation of relational content; a convolution-based bidirectional interaction module based on a convolutional operation, capturing pairwise bidirectional interactions of entity-relation, entity-qualifier, and relation-qualifier. realize the depth perception of the content related to the current statement. Furthermore, we introduce a Mixture-of-Experts strategy into the feed-forward layers of HyperFormer to strengthen its representation capabilities while reducing the amount of model parameters and computation. Extensive experiments on three well-known datasets with four different conditions demonstrate HyperFormer's effectiveness. Datasets and code are available at https://github.com/zhiweihu1103/HKGC-HyperFormer.