mlsanityData to 7 Oct 2026

Knowledge Generation -- Variational Bayes on Knowledge Graphs

arXiv 2101.08857 ↗First posted 21 Jan 2021cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

This thesis is a proof of concept for the potential of Variational Auto-Encoder (VAE) on representation learning of real-world Knowledge Graphs (KG). Inspired by successful approaches to the generation of molecular graphs, we evaluate the capabilities of our model, the Relational Graph Variational Auto-Encoder (RGVAE). The impact of the modular hyperparameter choices, encoding through graph convolutions, graph matching and latent space prior, is compared. The RGVAE is first evaluated on link prediction. The mean reciprocal rank (MRR) scores on the two datasets FB15K-237 and WN18RR are compared to the embedding-based model DistMult. A variational DistMult and a RGVAE without latent space prior constraint are implemented as control models. The results show that between different settings, the RGVAE with relaxed latent space, scores highest on both datasets, yet does not outperform the DistMult. Further, we investigate the latent space in a twofold experiment: first, linear interpolation between the latent representation of two triples, then the exploration of each latent dimension in a 95% confidence interval. Both interpolations show that the RGVAE learns to reconstruct the adjacency matrix but fails to disentangle. For the last experiment we introduce a new validation method for the FB15K-237 data set. The relation type-constrains of generated triples are filtered and matched with entity types. The observed rate of valid generated triples is insignificantly higher than the random threshold. All generated and valid triples are unseen. A comparison between different latent space priors, using the δ-VAE method, reveals a decoder collapse. Finally we analyze the limiting factors of our approach compared to molecule generation and propose solutions for the decoder collapse and successful representation learning of multi-relational KGs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 74 references that are in the map. First posted on arXiv 21 Jan 2021.

Builds on

23 of 74 references.

References · 23

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  5. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  6. 2016Variational Graph Auto-Encoderscited by 1,373
  7. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  8. 2019An Introduction to Variational Autoencoderscited by 997
  9. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 929
  10. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  11. 2017Convolutional 2D Knowledge Graph Embeddingscited by 729
  12. 2018GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoderscited by 375
11 more
  1. 2017Wasserstein Auto-Encoderscited by 371
  2. 2018AllenNLP: A Deep Semantic Natural Language Processing Platformcited by 347
  3. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  4. 2017Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphscited by 307
  5. 2015A Review of Relational Machine Learning for Knowledge Graphscited by 264
  6. 2019Contrastive Learning of Structured World Modelscited by 206
  7. 2016Variational Lossy Autoencodercited by 175
  8. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  9. 2020Gradient Centralization: A New Optimization Technique for Deep Neural Networkscited by 82
  10. 2018Avoiding Latent Variable Collapse With Generative Skip Modelscited by 47
  11. 2019Image-Conditioned Graph Generation for Road Network Extractioncited by 17

Built on by

Not cited yet.