# LayerDAG: A Layerwise Autoregressive Diffusion Model for Directed Acyclic Graph Generation arXiv: https://arxiv.org/abs/2411.02322 ยท HTML: https://mlsanity.com/paper/2411.02322/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. - Builds on 48 of its 73 references that are in the map. - First posted on arXiv 4 Nov 2024. ## Placed in - Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/) ## Builds on (most cited first, 30 of 48) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022) - [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021) - [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021) - [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015) - [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021) - [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018) - [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021) - [Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning](https://mlsanity.com/paper/2208.06193/) (2022) - [DiGress: Discrete Denoising diffusion for graph generation](https://mlsanity.com/paper/2209.14734/) (2022) - [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020) - [GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models](https://mlsanity.com/paper/1802.08773/) (2018) - [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2101.12072/) (2021) - [Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations](https://mlsanity.com/paper/2202.02514/) (2022) - [Permutation Invariant Graph Generation via Score-Based Generative Modeling](https://mlsanity.com/paper/2003.00638/) (2020) - [Imitating Human Behaviour with Diffusion Models](https://mlsanity.com/paper/2301.10677/) (2023) - [Efficient Graph Generation with Graph Recurrent Attention Networks](https://mlsanity.com/paper/1910.00760/) (2019) - [Learning Deep Generative Models of Graphs](https://mlsanity.com/paper/1803.03324/) (2018) - [Autoregressive Diffusion Models](https://mlsanity.com/paper/2110.02037/) (2021) - [Exploring Length Generalization in Large Language Models](https://mlsanity.com/paper/2207.04901/) (2022) - [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020) - [What Algorithms can Transformers Learn? A Study in Length Generalization](https://mlsanity.com/paper/2310.16028/) (2023) - [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018) - [Neural Predictor for Neural Architecture Search](https://mlsanity.com/paper/1912.00848/) (2019) - [Randomized Positional Encodings Boost Length Generalization of Transformers](https://mlsanity.com/paper/2305.16843/) (2023) - [D-VAE: A Variational Autoencoder for Directed Acyclic Graphs](https://mlsanity.com/paper/1904.11088/) (2019) ## Built on by (most cited first, 8 of 8) - [Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spaces](https://mlsanity.com/paper/2506.07903/) (2025) - [MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control](https://mlsanity.com/paper/2508.10684/) (2025) - [Relational In-Context Learning via Synthetic Pre-training with Structural Prior](https://mlsanity.com/paper/2603.03805/) (2026) - [Q-Fusion: Diffusing Quantum Circuits](https://mlsanity.com/paper/2504.20794/) (2025) - [Principled Latent Diffusion for Graphs via Laplacian Autoencoders](https://mlsanity.com/paper/2601.13780/) (2026) - [Generating Directed Graphs with Dual Attention and Asymmetric Encoding](https://mlsanity.com/paper/2506.16404/) (2025) - [ArtNet: Hierarchical Clustering-Based Artificial Netlist Generator for ML and DTCO Application](https://mlsanity.com/paper/2510.13582/) (2025) - [GeoGAE: Scalable Graph-Level Autoencoding via Hyperball Cloud Representations](https://mlsanity.com/paper/2609.35527/) (2026) ## Abstract (arXiv) Directed acyclic graphs (DAGs) serve as crucial data representations in domains such as hardware synthesis and compiler/program optimization for computing systems. DAG generative models facilitate the creation of synthetic DAGs, which can be used for benchmarking computing systems while preserving intellectual property. However, generating realistic DAGs is challenging due to their inherent directional and logical dependencies. This paper introduces LayerDAG, an autoregressive diffusion model, to address these challenges. LayerDAG decouples the strong node dependencies into manageable units that can be processed sequentially. By interpreting the partial order of nodes as a sequence of bipartite graphs, LayerDAG leverages autoregressive generation to model directional dependencies and employs diffusion models to capture logical dependencies within each bipartite graph. Comparative analyses demonstrate that LayerDAG outperforms existing DAG generative models in both expressiveness and generalization, particularly for generating large-scale DAGs with up to 400 nodes-a critical scenario for system benchmarking. Extensive experiments on both synthetic and real-world flow graphs from various computing platforms show that LayerDAG generates valid DAGs with superior statistical properties and benchmarking performance. The synthetic DAGs generated by LayerDAG enhance the training of ML-based surrogate models, resulting in improved accuracy in predicting performance metrics of real-world DAGs across diverse computing platforms.