# SwinGNN: Rethinking Permutation Invariance in Diffusion Models for Graph Generation

arXiv: https://arxiv.org/abs/2307.01646 · HTML: https://mlsanity.com/paper/2307.01646/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 54 of its 80 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks](https://mlsanity.com/paper/1810.02244/) (2018)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [DiGress: Discrete Denoising diffusion for graph generation](https://mlsanity.com/paper/2209.14734/) (2022)
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](https://mlsanity.com/paper/2203.02923/) (2022)
- [GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models](https://mlsanity.com/paper/1802.08773/) (2018)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [Provably Powerful Graph Networks](https://mlsanity.com/paper/1905.11136/) (2019)
- [GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders](https://mlsanity.com/paper/1802.03480/) (2018)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [Invariant and Equivariant Graph Networks](https://mlsanity.com/paper/1812.09902/) (2018)
- [GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation](https://mlsanity.com/paper/2001.09382/) (2020)

## Built on by (most cited first, 28 of 28)

- [Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series](https://mlsanity.com/paper/2410.19538/) (2024)
- [Efficient and Scalable Graph Generation through Iterative Local Expansion](https://mlsanity.com/paper/2312.11529/) (2023)
- [MoFlow: One-Step Flow Matching for Human Trajectory Forecasting via Implicit Maximum Likelihood Estimation based Distillation](https://mlsanity.com/paper/2503.09950/) (2025)
- [Pard: Permutation-Invariant Autoregressive Diffusion for Graph Generation](https://mlsanity.com/paper/2402.03687/) (2024)
- [Generative Modelling of Structurally Constrained Graphs](https://mlsanity.com/paper/2406.17341/) (2024)
- [A Simple and Scalable Representation for Graph Generation](https://mlsanity.com/paper/2312.02230/) (2023)
- [Improving Molecular Graph Generation with Flow Matching and Optimal Transport](https://mlsanity.com/paper/2411.05676/) (2024)
- [Equivariant Denoisers Cannot Copy Graphs: Align Your Graph Diffusion Models](https://mlsanity.com/paper/2405.17656/) (2024)
- [TrajFlow: Multi-modal Motion Prediction via Flow Matching](https://mlsanity.com/paper/2506.08541/) (2025)
- [FragFM: Hierarchical Framework for Efficient Molecule Generation via Fragment-Level Discrete Flow Matching](https://mlsanity.com/paper/2502.15805/) (2025)
- [Rethinking Diffusion Models with Symmetries through Canonicalization with Applications to Molecular Graph Generation](https://mlsanity.com/paper/2602.15022/) (2026)
- [Reviving Life on the Edge: Joint Score-Based Graph Generation of Rich Edge Attributes](https://mlsanity.com/paper/2402.04046/) (2024)
- [Generating Physical Dynamics under Priors](https://mlsanity.com/paper/2409.00730/) (2024)
- [RETRO SYNFLOW: Discrete Flow Matching for Accurate and Diverse Single-Step Retrosynthesis](https://mlsanity.com/paper/2506.04439/) (2025)
- [Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Models](https://mlsanity.com/paper/2401.01130/) (2024)
- [DDPS: Discrete Diffusion Posterior Sampling for Paths in Layered Graphs](https://mlsanity.com/paper/2504.20754/) (2025)
- [Diffusion-Free Graph Generation with Next-Scale Prediction](https://mlsanity.com/paper/2503.23612/) (2025)
- [Generative AI-Based Probabilistic Constellation Shaping With Diffusion Models](https://mlsanity.com/paper/2311.09349/) (2023)
- [ACDZero: MCTS Agent for Mastering Automated Cyber Defense](https://mlsanity.com/paper/2601.02196/) (2026)
- [SeaDAG: Semi-autoregressive Diffusion for Conditional Directed Acyclic Graph Generation](https://mlsanity.com/paper/2410.16119/) (2024)
- [Graph VQ-Transformer (GVT): Fast and Accurate Molecular Generation via High-Fidelity Discrete Latents](https://mlsanity.com/paper/2512.02667/) (2025)
- [Subgraph Generation for Generalizing on Out-of-Distribution Links](https://mlsanity.com/paper/2507.11710/) (2025)
- [Balancing Symmetry and Efficiency in Graph Flow Matching](https://mlsanity.com/paper/2602.18084/) (2026)
- [Modular Diffusion Models for Structured Visual Recognition](https://mlsanity.com/paper/2606.22702/) (2026)
- [From topology learning to graph generation: A unifying perspective](https://mlsanity.com/paper/2609.02286/) (2026)
- [GLASS: Global Latent Aggregation with Slot-based Set Decoding for Scalable All-Atom Crystal Generation](https://mlsanity.com/paper/2609.37158/) (2026)
- [Consistent Distribution Matching for Data-Free Diffusion Distillation](https://mlsanity.com/paper/2610.09221/) (2026)
- [Your Discrete Graph Diffusion Model is Secretly Memorizing](https://mlsanity.com/paper/2610.13786/) (2026)

## Abstract (arXiv)

Diffusion models based on permutation-equivariant networks can learn permutation-invariant distributions for graph data. However, in comparison to their non-invariant counterparts, we have found that these invariant models encounter greater learning challenges since 1) their effective target distributions exhibit more modes; 2) their optimal one-step denoising scores are the score functions of Gaussian mixtures with more components. Motivated by this analysis, we propose a non-invariant diffusion model, called textitSwinGNN, which employs an efficient edge-to-edge 2-WL message passing network and utilizes shifted window based self-attention inspired by SwinTransformers. Further, through systematic ablations, we identify several critical training and sampling techniques that significantly improve the sample quality of graph generation. At last, we introduce a simple post-processing trick, textiti.e., randomly permuting the generated graphs, which provably converts any graph generative model to a permutation-invariant one. Extensive experiments on synthetic and real-world protein and molecule datasets show that our SwinGNN achieves state-of-the-art performances. Our code is released at https://github.com/qiyan98/SwinGNN.