# A Graph VAE and Graph Transformer Approach to Generating Molecular Graphs

arXiv: https://arxiv.org/abs/2104.04345 · HTML: https://mlsanity.com/paper/2104.04345/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 9 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Hierarchical Graph Representation Learning with Differentiable Pooling](https://mlsanity.com/paper/1806.08804/) (2018)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders](https://mlsanity.com/paper/1802.03480/) (2018)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [Disentangling Interpretable Generative Parameters of Random and Real-World Graphs](https://mlsanity.com/paper/1910.05639/) (2019)

## Built on by (most cited first, 16 of 16)

- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [SPECTRE: Spectral Conditioning Helps to Overcome the Expressivity Limits of One-shot Graph Generators](https://mlsanity.com/paper/2204.01613/) (2022)
- [Discrete-state Continuous-time Diffusion for Graph Generation](https://mlsanity.com/paper/2405.11416/) (2024)
- [Top-N: Equivariant set and graph generation without exchangeability](https://mlsanity.com/paper/2110.02096/) (2021)
- [Coarse-to-Fine: a Hierarchical Diffusion Model for Molecule Generation in 3D](https://mlsanity.com/paper/2305.13266/) (2023)
- [Optimizing OOD Detection in Molecular Graphs: A Novel Approach with Diffusion Models](https://mlsanity.com/paper/2404.15625/) (2024)
- [Brain Multigraph Prediction using Topology-Aware Adversarial Graph Neural Network](https://mlsanity.com/paper/2105.02565/) (2021)
- [Target Specific De Novo Design of Drug Candidate Molecules with Graph Transformer-based Generative Adversarial Networks](https://mlsanity.com/paper/2302.07868/) (2023)
- [Diffusion on Graph: Augmentation of Graph Structure for Node Classification](https://mlsanity.com/paper/2503.12563/) (2025)
- [GGBall: Graph Generative Model on Poincar'e Ball](https://mlsanity.com/paper/2506.07198/) (2025)
- [Interpreting Equivariant Representations](https://mlsanity.com/paper/2401.12588/) (2024)
- [DMol: A Highly Efficient and Chemical Motif-Preserving Molecule Generation Platform](https://mlsanity.com/paper/2504.06312/) (2025)
- [SFi-Former: Sparse Flow Induced Attention for Graph Transformer](https://mlsanity.com/paper/2504.20666/) (2025)
- [Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image](https://mlsanity.com/paper/2111.13023/) (2021)
- [JTreeformer: Graph-Transformer via Latent-Diffusion Model for Molecular Generation](https://mlsanity.com/paper/2504.20770/) (2025)
- [A Systematic Evaluation of Molecule Generation Models for De Novo Drug Design: From Benchmarks to Practical Insights](https://mlsanity.com/paper/2609.10099/) (2026)

## Abstract (arXiv)

We propose a combination of a variational autoencoder and a transformer based model which fully utilises graph convolutional and graph pooling layers to operate directly on graphs. The transformer model implements a novel node encoding layer, replacing the position encoding typically used in transformers, to create a transformer with no position information that operates on graphs, encoding adjacent node properties into the edge generation process. The proposed model builds on graph generative work operating on graphs with edge features, creating a model that offers improved scalability with the number of nodes in a graph. In addition, our model is capable of learning a disentangled, interpretable latent space that represents graph properties through a mapping between latent variables and graph properties. In experiments we chose a benchmark task of molecular generation, given the importance of both generated node and edge features. Using the QM9 dataset we demonstrate that our model performs strongly across the task of generating valid, unique and novel molecules. Finally, we demonstrate that the model is interpretable by generating molecules controlled by molecular properties, and we then analyse and visualise the learned latent representation.