# How to Turn Your Knowledge Graph Embeddings into Generative Models

arXiv: https://arxiv.org/abs/2305.15944 · HTML: https://mlsanity.com/paper/2305.15944/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 78 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015)
- [Canonical Tensor Decomposition for Knowledge Base Completion](https://mlsanity.com/paper/1806.07297/) (2018)
- [Semantic Probabilistic Layers for Neuro-Symbolic Learning](https://mlsanity.com/paper/2206.00426/) (2022)
- [Complex Query Answering with Neural Link Predictors](https://mlsanity.com/paper/2011.03459/) (2020)
- [A Semantic Matching Energy Function for Learning with Multi-relational Data](https://mlsanity.com/paper/1301.3485/) (2013)
- [On the Latent Variable Interpretation in Sum-Product Networks](https://mlsanity.com/paper/1601.06180/) (2016)
- [Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations](https://mlsanity.com/paper/2110.02834/) (2021)
- [Improving Knowledge Graph Embedding Using Simple Constraints](https://mlsanity.com/paper/1805.02408/) (2018)
- [Type-Constrained Representation Learning in Knowledge Graphs](https://mlsanity.com/paper/1508.02593/) (2015)
- [Tensor-Train Density Estimation](https://mlsanity.com/paper/2108.00089/) (2021)
- [TransG : A Generative Mixture Model for Knowledge Graph Embedding](https://mlsanity.com/paper/1509.05488/) (2015)
- [Strudel: Learning Structured-Decomposable Probabilistic Circuits](https://mlsanity.com/paper/2007.09331/) (2020)
- [Tractability through Exchangeability: A New Perspective on Efficient Probabilistic Inference](https://mlsanity.com/paper/1401.1247/) (2014)
- [Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning](https://mlsanity.com/paper/2104.04597/) (2021)
- [Probability Calibration for Knowledge Graph Embedding Models](https://mlsanity.com/paper/1912.10000/) (2019)
- [Visualizing and Understanding Sum-Product Networks](https://mlsanity.com/paper/1608.08266/) (2016)
- [Understanding the Performance of Knowledge Graph Embeddings in Drug Discovery](https://mlsanity.com/paper/2105.10488/) (2021)
- [Adversarial Sets for Regularising Neural Link Predictors](https://mlsanity.com/paper/1707.07596/) (2017)
- [Knowledge Base Completion: Baseline strikes back (Again)](https://mlsanity.com/paper/2005.00804/) (2020)
- [Learning with Memory Embeddings](https://mlsanity.com/paper/1511.07972/) (2015)
- [The Tensor Brain: A Unified Theory of Perception, Memory and Semantic Decoding](https://mlsanity.com/paper/2109.13392/) (2021)

## Built on by (most cited first, 20 of 20)

- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [Scaling Tractable Probabilistic Circuits: A Systems Perspective](https://mlsanity.com/paper/2406.00766/) (2024)
- [Image Inpainting via Tractable Steering of Diffusion Models](https://mlsanity.com/paper/2401.03349/) (2023)
- [What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?](https://mlsanity.com/paper/2409.07953/) (2024)
- [Sum of Squares Circuits](https://mlsanity.com/paper/2408.11778/) (2024)
- [Knowledge Graph Embeddings in the Biomedical Domain: Are They Useful? A Look at Link Prediction, Rule Learning, and Downstream Polypharmacy Tasks](https://mlsanity.com/paper/2305.19979/) (2023)
- [Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits](https://mlsanity.com/paper/2406.06494/) (2024)
- [Is Complex Query Answering Really Complex?](https://mlsanity.com/paper/2410.12537/) (2024)
- [How to Square Tensor Networks and Circuits Without Squaring Them](https://mlsanity.com/paper/2512.17090/) (2025)
- [Tractable Representation Learning with Probabilistic Circuits](https://mlsanity.com/paper/2507.04385/) (2025)
- [Improving Molecule Generation and Drug Discovery with a Knowledge-enhanced Generative Model](https://mlsanity.com/paper/2402.08790/) (2024)
- [On Faster Marginalization with Squared Circuits via Orthonormalization](https://mlsanity.com/paper/2412.07883/) (2024)
- [DuetGraph: Coarse-to-Fine Knowledge Graph Reasoning with Dual-Pathway Global-Local Fusion](https://mlsanity.com/paper/2507.11229/) (2025)
- [Temporal Smoothness Regularisers for Neural Link Predictors](https://mlsanity.com/paper/2309.09045/) (2023)
- [Probabilistic Graph Circuits: Deep Generative Models for Tractable Probabilistic Inference over Graphs](https://mlsanity.com/paper/2503.12162/) (2025)
- [Squared families: Searching beyond regular probability models](https://mlsanity.com/paper/2503.21128/) (2025)
- [On the Theoretical Limitations of Embedding-based Link Prediction](https://mlsanity.com/paper/2506.22271/) (2025)
- [Unifying Post-hoc Explanations of Knowledge Graph Completions](https://mlsanity.com/paper/2507.22951/) (2025)
- [How to Approximate Inference with Subtractive Mixture Models](https://mlsanity.com/paper/2604.16714/) (2026)
- [Autoregressive Models for Knowledge Graph Generation](https://mlsanity.com/paper/2602.06707/) (2026)

## Abstract (arXiv)

Some of the most successful knowledge graph embedding (KGE) models for link prediction -- CP, RESCAL, TuckER, ComplEx -- can be interpreted as energy-based models. Under this perspective they are not amenable for exact maximum-likelihood estimation (MLE), sampling and struggle to integrate logical constraints. This work re-interprets the score functions of these KGEs as circuits -- constrained computational graphs allowing efficient marginalisation. Then, we design two recipes to obtain efficient generative circuit models by either restricting their activations to be non-negative or squaring their outputs. Our interpretation comes with little or no loss of performance for link prediction, while the circuits framework unlocks exact learning by MLE, efficient sampling of new triples, and guarantee that logical constraints are satisfied by design. Furthermore, our models scale more gracefully than the original KGEs on graphs with millions of entities.