mlsanityData to 7 Oct 2026

LayoutDM: Transformer-based Diffusion Model for Layout Generation

arXiv 2305.02567 ↗First posted 4 May 2023cs.CVcited by 57

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Automatic layout generation that can synthesize high-quality layouts is an important tool for graphic design in many applications. Though existing methods based on generative models such as Generative Adversarial Networks (GANs) and Variational Auto-Encoders (VAEs) have progressed, they still leave much room for improving the quality and diversity of the results. Inspired by the recent success of diffusion models in generating high-quality images, this paper explores their potential for conditional layout generation and proposes Transformer-based Layout Diffusion Model (LayoutDM) by instantiating the conditional denoising diffusion probabilistic model (DDPM) with a purely transformer-based architecture. Instead of using convolutional neural networks, a transformer-based conditional Layout Denoiser is proposed to learn the reverse diffusion process to generate samples from noised layout data. Benefitting from both transformer and DDPM, our LayoutDM is of desired properties such as high-quality generation, strong sample diversity, faithful distribution coverage, and stationary training in comparison to GANs and VAEs. Quantitative and qualitative experimental results show that our method outperforms state-of-the-art generative models in terms of quality and diversity.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 36 of its 48 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

36 of 48 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
24 more
  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2016Layer Normalizationcited by 5,579
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  8. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  9. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  10. 2019PubLayNet: largest dataset ever for document layout analysiscited by 349
  11. 2021Generating Images with Sparse Representationscited by 262
  12. 2020LayoutTransformer: Layout Generation and Completion with Self-attentioncited by 122
  13. 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
  14. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  15. 2021CanvasVAE: Learning to Generate Vector Graphic Documentscited by 93
  16. 2021Variational Transformer Networks for Layout Generationcited by 88
  17. 2019Neural Design Network: Graphic Layout Generation with Constraintscited by 83
  18. 2021Constrained Graphic Layout Generation via Latent Optimizationcited by 83
  19. 2019Specifying Object Attributes and Relations in Interactive Scene Generationcited by 76
  20. 2021BLT: Bidirectional Layout Transformer for Controllable Layout Generationcited by 69
  21. 2020Attribute-conditioned Layout GAN for Automatic Graphic Designcited by 67
  22. 2019READ: Recursive Autoencoders for Document Layout Generationcited by 51
  23. 2022Geometry Aligned Variational Transformer for Image-conditioned Layout Generationcited by 33
  24. 2022Aesthetic Text Logo Synthesis via Content-aware Layout Inferringcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 24; 2025: 18; 2026: 9.

57 papers.

Sort:

By area:

  1. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  2. 2025ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generationcited by 37
  3. 2023Retrieval-Augmented Layout Transformer for Content-Aware Layout Generationcited by 34
  4. 2024PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generationcited by 34
  5. 2023UGG: Unified Generative Graspingcited by 30
  6. 2024Mixed Diffusion for 3D Indoor Scene Synthesiscited by 30
  7. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  8. 2024Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraintscited by 26
  9. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  10. 2024LayoutFlow: Flow Matching for Layout Generationcited by 21
  11. 2024Alfie: Democratising RGBA Image Generation With No cited by 20
  12. 2023LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Modelscited by 17
38 more
  1. 2023Planning and Rendering: Towards Product Poster Generation with Diffusion Modelscited by 15
  2. 2025PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generationcited by 15
  3. 2024SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posterscited by 14
  4. 2024VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correctioncited by 14
  5. 2024Training-free Composite Scene Generation for Layout-to-Image Synthesiscited by 11
  6. 2024TKG-DM: Training-free Chroma Key Content Generation Diffusion Modelcited by 11
  7. 2024COHO: Context-Sensitive City-Scale Hierarchical Urban Layout Generationcited by 10
  8. 2024LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generationcited by 10
  9. 2024Towards Better Text-to-Image Generation Alignment via Attention Modulationcited by 9
  10. 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Modelscited by 8
  11. 2024Layout-Corrector: Alleviating Layout Sticking Phenomenon in Discrete Diffusion Modelcited by 7
  12. 2025CAL-RAG: Retrieval-Augmented Multi-Agent Generation for Content-Aware Layout Designcited by 7
  13. 2024InstructLayout: Instruction-Driven 2D and 3D Layout Synthesis with Semantic Graph Priorcited by 6
  14. 2025Lay-Your-Scene: Natural Scene Layout Generation with Diffusion Transformerscited by 6
  15. 2025SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesiscited by 6
  16. 2024Spatial Diffusion for Cell Layout Generationcited by 5
  17. 2025Smaller But Better: Unifying Layout Generation with Smaller Large Language Modelscited by 5
  18. 2024Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Modelscited by 4
  19. 2024Imagining the Unseen: Generative Location Modeling for Object Placementcited by 4
  20. 2026Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligencecited by 4
  21. 2024TextCenGen: Attention-Guided Text-Centric Background Adaptation for Text-to-Image Generationcited by 3
  22. 2025LayoutRAG: Retrieval-Augmented Model for Content-agnostic Conditional Layout Generationcited by 3
  23. 2025From Fragment to One Piece: A Survey on AI-Driven Graphic Designcited by 2
  24. 2025Scan-and-Print: Patch-level Data Summarization and Augmentation for Content-aware Layout Generation in Poster Designcited by 2
  25. 2024Learning to Kern: Set-wise Estimation of Optimal Letter Spacecited by 1
  26. 2024LayoutDiT: Exploring Content-Graphic Balance in Layout Generation with Diffusion Transformercited by 1
  27. 2024Diffusion-Inspired Cold Start with Sufficient Prior in Computerized Adaptive Testingcited by 1
  28. 2025DesignLab: Designing Slides Through Iterative Detection and Correctioncited by 1
  29. 2025LLMs as Layout Designers: Enhanced Spatial Reasoning for Content-Aware Layout Generationcited by 1
  30. 2026Bridging Gulfs in UI Generation through Semantic Guidancecited by 1
  31. 2026From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Designcited by 1
  32. 2026PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garmentscited by 1
  33. 2026Unveiling the Secret of AdaLN-Zero in Diffusion Transformercited by 1
  34. 2024Improving Fine-Grained Control via Aggregation of Multiple Diffusion Modelscited by 0
  35. 2024Targeted Therapy in Data Removal: Object Unlearning Based on Scene Graphscited by 0
  36. 2025Generating Animated Layouts as Structured Text Representationscited by 0
  37. 2025Few-Shot Anomaly-Driven Generation for Anomaly Classification and Segmentationcited by 0
  38. 2025SMART-Editor: A Multi-Agent Framework for Human-Like Design Editing with Structural Integritycited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Models
  2. 2024DogLayout: Denoising Diffusion GAN for Discrete and Continuous Layout Generation
  3. 2023DLT: Conditioned layout generation with Joint Discrete-Continuous Diffusion Layout Transformer
  4. 2023LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation
  5. 2023Unifying Layout Generation with a Decoupled Diffusion Model
  6. 2025UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation
  7. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generation
  8. 2024Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive