# DLT: Conditioned layout generation with Joint Discrete-Continuous Diffusion Layout Transformer

arXiv: https://arxiv.org/abs/2303.03755 · HTML: https://mlsanity.com/paper/2303.03755/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 36 of its 41 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [DiffusionDet: Diffusion Model for Object Detection](https://mlsanity.com/paper/2211.09788/) (2022)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models](https://mlsanity.com/paper/2301.12661/) (2023)
- [Diffsound: Discrete Diffusion Model for Text-to-sound Generation](https://mlsanity.com/paper/2207.09983/) (2022)
- [GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium](https://mlsanity.com/paper/1706.08500/) (2017)
- [LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators](https://mlsanity.com/paper/1901.06767/) (2019)
- [LayoutVAE: Stochastic Scene Layout Generation From a Label Set](https://mlsanity.com/paper/1907.10719/) (2019)
- [Variational Transformer Networks for Layout Generation](https://mlsanity.com/paper/2104.02416/) (2021)
- [Neural Design Network: Graphic Layout Generation with Constraints](https://mlsanity.com/paper/1912.09421/) (2019)
- [Constrained Graphic Layout Generation via Latent Optimization](https://mlsanity.com/paper/2108.00871/) (2021)
- [BLT: Bidirectional Layout Transformer for Controllable Layout Generation](https://mlsanity.com/paper/2112.05112/) (2021)

## Built on by (most cited first, 20 of 20)

- [Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation](https://mlsanity.com/paper/2311.13602/) (2023)
- [Dolfin: Diffusion Layout Transformers without Autoencoder](https://mlsanity.com/paper/2310.16305/) (2023)
- [LayoutFlow: Flow Matching for Layout Generation](https://mlsanity.com/paper/2403.18187/) (2024)
- [Boosting GUI Prototyping with Diffusion Models](https://mlsanity.com/paper/2306.06233/) (2023)
- [LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generation](https://mlsanity.com/paper/2407.12356/) (2024)
- [Layout-Corrector: Alleviating Layout Sticking Phenomenon in Discrete Diffusion Model](https://mlsanity.com/paper/2409.16689/) (2024)
- [Remix-DiT: Mixing Diffusion Transformers for Multi-Expert Denoising](https://mlsanity.com/paper/2412.05628/) (2024)
- [SketchDNN: Joint Continuous-Discrete Diffusion for CAD Sketch Generation](https://mlsanity.com/paper/2507.11579/) (2025)
- [JointDiff: Bridging Continuous and Discrete in Multi-Agent Trajectory Generation](https://mlsanity.com/paper/2509.22522/) (2025)
- [Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Models](https://mlsanity.com/paper/2401.01130/) (2024)
- [LayoutRAG: Retrieval-Augmented Model for Content-agnostic Conditional Layout Generation](https://mlsanity.com/paper/2506.02697/) (2025)
- [SLayR: Scene Layout Generation with Rectified Flow](https://mlsanity.com/paper/2412.05003/) (2024)
- [From Fragment to One Piece: A Survey on AI-Driven Graphic Design](https://mlsanity.com/paper/2503.18641/) (2025)
- [Agentic Design Review System](https://mlsanity.com/paper/2508.10745/) (2025)
- [LayoutDiT: Exploring Content-Graphic Balance in Layout Generation with Diffusion Transformer](https://mlsanity.com/paper/2407.15233/) (2024)
- [Interleaved Gibbs Diffusion: Generating Discrete-Continuous Data with Implicit Constraints](https://mlsanity.com/paper/2502.13450/) (2025)
- [Unveiling the Secret of AdaLN-Zero in Diffusion Transformer](https://mlsanity.com/paper/2608.09438/) (2026)
- [Enhancing Retinal Vessel Segmentation Generalization via Layout-Aware Generative Modelling](https://mlsanity.com/paper/2503.01190/) (2025)
- [Step-by-step Layered Design Generation](https://mlsanity.com/paper/2512.03335/) (2025)
- [Mixed neural posterior estimation for simulators with discrete and continuous parameters](https://mlsanity.com/paper/2605.13551/) (2026)

## Abstract (arXiv)

Generating visual layouts is an essential ingredient of graphic design. The ability to condition layout generation on a partial subset of component attributes is critical to real-world applications that involve user interaction. Recently, diffusion models have demonstrated high-quality generative performances in various domains. However, it is unclear how to apply diffusion models to the natural representation of layouts which consists of a mix of discrete (class) and continuous (location, size) attributes. To address the conditioning layout generation problem, we introduce DLT, a joint discrete-continuous diffusion model. DLT is a transformer-based model which has a flexible conditioning mechanism that allows for conditioning on any given subset of all the layout component classes, locations, and sizes. Our method outperforms state-of-the-art generative models on various layout generation datasets with respect to different metrics and conditioning settings. Additionally, we validate the effectiveness of our proposed conditioning mechanism and the joint continuous-diffusion process. This joint process can be incorporated into a wide range of mixed discrete-continuous generative tasks.