# Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints

arXiv: https://arxiv.org/abs/2402.04754 · HTML: https://mlsanity.com/paper/2402.04754/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [LayoutDM: Discrete Diffusion Model for Controllable Layout Generation](https://mlsanity.com/paper/2303.08137/) (2023)
- [LayoutTransformer: Layout Generation and Completion with Self-attention](https://mlsanity.com/paper/2006.14615/) (2020)
- [LayoutVAE: Stochastic Scene Layout Generation From a Label Set](https://mlsanity.com/paper/1907.10719/) (2019)
- [CARD: Classification and Regression Diffusion Models](https://mlsanity.com/paper/2206.07275/) (2022)
- [Variational Transformer Networks for Layout Generation](https://mlsanity.com/paper/2104.02416/) (2021)
- [Neural Design Network: Graphic Layout Generation with Constraints](https://mlsanity.com/paper/1912.09421/) (2019)
- [Constrained Graphic Layout Generation via Latent Optimization](https://mlsanity.com/paper/2108.00871/) (2021)
- [Diffusion Models as Masked Autoencoders](https://mlsanity.com/paper/2304.03283/) (2023)
- [BLT: Bidirectional Layout Transformer for Controllable Layout Generation](https://mlsanity.com/paper/2112.05112/) (2021)
- [Attribute-conditioned Layout GAN for Automatic Graphic Design](https://mlsanity.com/paper/2009.05284/) (2020)
- [LayoutDM: Transformer-based Diffusion Model for Layout Generation](https://mlsanity.com/paper/2305.02567/) (2023)
- [LayoutFormer++: Conditional Graphic Layout Generation via Constraint Serialization and Decoding Space Restriction](https://mlsanity.com/paper/2208.08037/) (2022)
- [Unifying Layout Generation with a Decoupled Diffusion Model](https://mlsanity.com/paper/2303.05049/) (2023)
- [LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Models](https://mlsanity.com/paper/2303.11589/) (2023)
- [PLay: Parametrically Conditioned Layout Generation using Latent Diffusion](https://mlsanity.com/paper/2301.11529/) (2023)
- [Diffusion-based Document Layout Generation](https://mlsanity.com/paper/2303.10787/) (2023)

## Built on by (most cited first, 26 of 26)

- [DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perception](https://mlsanity.com/paper/2410.12628/) (2024)
- [LayoutFlow: Flow Matching for Layout Generation](https://mlsanity.com/paper/2403.18187/) (2024)
- [Alfie: Democratising RGBA Image Generation With No ](https://mlsanity.com/paper/2408.14826/) (2024)
- [Constrained Diffusion Models via Dual Training](https://mlsanity.com/paper/2408.15094/) (2024)
- [VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correction](https://mlsanity.com/paper/2412.04237/) (2024)
- [Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluation](https://mlsanity.com/paper/2508.02374/) (2025)
- [Composition and Alignment of Diffusion Models using Constrained Learning](https://mlsanity.com/paper/2508.19104/) (2025)
- [AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models](https://mlsanity.com/paper/2503.00591/) (2025)
- [TextLap: Customizing Language Models for Text-to-Layout Planning](https://mlsanity.com/paper/2410.12844/) (2024)
- [Layout-Corrector: Alleviating Layout Sticking Phenomenon in Discrete Diffusion Model](https://mlsanity.com/paper/2409.16689/) (2024)
- [Design-o-meter: Towards Evaluating and Refining Graphic Designs](https://mlsanity.com/paper/2411.14959/) (2024)
- [SciPostGen: Bridging the Gap between Scientific Papers and Poster Layouts](https://mlsanity.com/paper/2511.22490/) (2025)
- [LayoutRAG: Retrieval-Augmented Model for Content-agnostic Conditional Layout Generation](https://mlsanity.com/paper/2506.02697/) (2025)
- [Style Customization of Text-to-Vector Generation with Image Diffusion Priors](https://mlsanity.com/paper/2505.10558/) (2025)
- [Identity-Preserving Text-to-Image Generation via Dual-Level Feature Decoupling and Expert-Guided Fusion](https://mlsanity.com/paper/2505.22360/) (2025)
- [OmniDocLayout: Towards Diverse Document Layout Generation via Coarse-to-Fine LLM Learning](https://mlsanity.com/paper/2510.26213/) (2025)
- [PSDesigner: Automated Graphic Design with a Human-Like Creative Workflow](https://mlsanity.com/paper/2603.25738/) (2026)
- [LayoutDiT: Exploring Content-Graphic Balance in Layout Generation with Diffusion Transformer](https://mlsanity.com/paper/2407.15233/) (2024)
- [Interleaved Gibbs Diffusion: Generating Discrete-Continuous Data with Implicit Constraints](https://mlsanity.com/paper/2502.13450/) (2025)
- [DocReward: A Document Reward Model for Structuring and Stylizing](https://mlsanity.com/paper/2510.11391/) (2025)
- [Generating Animated Layouts as Structured Text Representations](https://mlsanity.com/paper/2505.00975/) (2025)
- [UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation](https://mlsanity.com/paper/2512.08897/) (2025)
- [DesignSense: A Human Preference Dataset and Reward Modeling Framework for Graphic Layout Generation](https://mlsanity.com/paper/2602.23438/) (2026)
- [Towards Design Compositing](https://mlsanity.com/paper/2604.14605/) (2026)
- [See Before You Code: Learning Visual Priors for Spatially Aware Educational Animation Generation](https://mlsanity.com/paper/2605.15585/) (2026)
- [Learned End-to-End Guidance Schedules for Diffusion Models](https://mlsanity.com/paper/2610.01502/) (2026)

## Abstract (arXiv)

Controllable layout generation refers to the process of creating a plausible visual arrangement of elements within a graphic design (e.g., document and web designs) with constraints representing design intentions. Although recent diffusion-based models have achieved state-of-the-art FID scores, they tend to exhibit more pronounced misalignment compared to earlier transformer-based models. In this work, we propose the LAyout Constraint diffusion modEl (LACE), a unified model to handle a broad range of layout generation tasks, such as arranging elements with specified attributes and refining or completing a coarse layout design. The model is based on continuous diffusion models. Compared with existing methods that use discrete diffusion models, continuous state-space design can enable the incorporation of differentiable aesthetic constraint functions in training. For conditional generation, we introduce conditions via masked input. Extensive experiment results show that LACE produces high-quality layouts and outperforms existing state-of-the-art baselines.