# Generative Representation Learning on Hyper-relational Knowledge Graphs via Masked Discrete Diffusion

arXiv: https://arxiv.org/abs/2605.24064 · HTML: https://mlsanity.com/paper/2605.24064/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 33 of its 58 references that are in the map.
- First posted on arXiv 22 May 2026.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data](https://mlsanity.com/paper/2406.03736/) (2024)
- [Message Passing for Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2009.10847/) (2020)
- [KGGen: Extracting Knowledge Graphs from Plain Text with Language Models](https://mlsanity.com/paper/2502.09956/) (2025)
- [KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion](https://mlsanity.com/paper/2402.02389/) (2024)
- [Role-Aware Modeling for N-ary Relational Knowledge Bases](https://mlsanity.com/paper/2104.09780/) (2021)
- [Link Prediction on N-ary Relational Facts: A Graph-based Approach](https://mlsanity.com/paper/2105.08476/) (2021)
- [Multi-perspective Improvement of Knowledge Graph Completion with Large Language Models](https://mlsanity.com/paper/2403.01972/) (2024)
- [Improving Hyper-Relational Knowledge Graph Completion](https://mlsanity.com/paper/2104.08167/) (2021)
- [MKGL: Mastery of a Three-Word Language](https://mlsanity.com/paper/2410.07526/) (2024)
- [Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers](https://mlsanity.com/paper/2305.18256/) (2023)
- [KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledge](https://mlsanity.com/paper/2405.16412/) (2024)
- [Link Prediction on N-ary Relational Data Based on Relatedness Evaluation](https://mlsanity.com/paper/2104.10424/) (2021)
- [Searching to Sparsify Tensor Decomposition for N-ary Relational Data](https://mlsanity.com/paper/2104.10625/) (2021)
- [Shrinking Embeddings for Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2306.02199/) (2023)
- [Text2NKG: Fine-Grained N-ary Relation Extraction for N-ary relational Knowledge Graph Construction](https://mlsanity.com/paper/2310.05185/) (2023)
- [Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models](https://mlsanity.com/paper/2501.18119/) (2025)
- [HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level](https://mlsanity.com/paper/2305.06588/) (2023)
- [Link Prediction with Relational Hypergraphs](https://mlsanity.com/paper/2402.04062/) (2024)
- [Start from Zero: Triple Set Prediction for Automatic Knowledge Graph Completion](https://mlsanity.com/paper/2406.18166/) (2024)
- [HyCubE: Efficient Knowledge Hypergraph 3D Circular Convolutional Embedding](https://mlsanity.com/paper/2402.08961/) (2024)
- [Incorporating Discriminator in Sentence Generation: a Gibbs Sampling Method](https://mlsanity.com/paper/1802.08970/) (2018)
- [HyperFormer: Enhancing Entity and Relation Interaction for Hyper-Relational Knowledge Graph Completion](https://mlsanity.com/paper/2308.06512/) (2023)

## Abstract (arXiv)

Hyper-relational knowledge graphs (HKGs) effectively represent complex facts. While inferring new knowledge in HKGs is a critical problem, current methods cast it as a simple link prediction, assuming that nearly all entities and relations within a fact are known, leaving only a single blank to be filled. However, this restricted assumption may not hold in real-world scenarios in which multiple, or even all, constituent components of a fact may be missing simultaneously. To bridge this gap, we introduce a task called fact generation: generating a valid hyper-relational fact from an arbitrarily masked query, i.e., completing a partially observed fact or generating a fact from scratch. We propose KREPE, the first generative representation learning method for HKGs that learns to model the probability distributions of missing components conditioned on the local fact components and global structure of HKGs via a masked discrete diffusion. KREPE models both the intra-fact dependencies by contextual message passing and inter-fact correlations by aggregating stochastically sampled contexts. KREPE seamlessly unifies link prediction and fact generation within a single training framework, achieving state-of-the-art performance on standard HKG link prediction benchmarks and outperforming LLM-based baselines in generating novel and correct facts.