# UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation

arXiv: https://arxiv.org/abs/2512.08897 · HTML: https://mlsanity.com/paper/2512.08897/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 9 Dec 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 35)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation](https://mlsanity.com/paper/2402.17245/) (2024)
- [In-Context LoRA for Diffusion Transformers](https://mlsanity.com/paper/2410.23775/) (2024)
- [LayoutDM: Discrete Diffusion Model for Controllable Layout Generation](https://mlsanity.com/paper/2303.08137/) (2023)
- [LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators](https://mlsanity.com/paper/1901.06767/) (2019)
- [LayoutTransformer: Layout Generation and Completion with Self-attention](https://mlsanity.com/paper/2006.14615/) (2020)
- [LayoutVAE: Stochastic Scene Layout Generation From a Label Set](https://mlsanity.com/paper/1907.10719/) (2019)
- [EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer](https://mlsanity.com/paper/2503.07027/) (2025)
- [Highly Accurate Dichotomous Image Segmentation](https://mlsanity.com/paper/2203.03041/) (2022)
- [CanvasVAE: Learning to Generate Vector Graphic Documents](https://mlsanity.com/paper/2108.01249/) (2021)
- [Constrained Graphic Layout Generation via Latent Optimization](https://mlsanity.com/paper/2108.00871/) (2021)
- [Attribute-conditioned Layout GAN for Automatic Graphic Design](https://mlsanity.com/paper/2009.05284/) (2020)
- [PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layout](https://mlsanity.com/paper/2303.15937/) (2023)
- [CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generation](https://mlsanity.com/paper/2412.03859/) (2024)
- [Composition-aware Graphic Layout GAN for Visual-textual Presentation Designs](https://mlsanity.com/paper/2205.00303/) (2022)
- [LayoutPrompter: Awaken the Design Ability of Large Language Models](https://mlsanity.com/paper/2311.06495/) (2023)
- [OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows](https://mlsanity.com/paper/2412.01169/) (2024)
- [LayoutFormer++: Conditional Graphic Layout Generation via Constraint Serialization and Decoding Space Restriction](https://mlsanity.com/paper/2208.08037/) (2022)
- [Unifying Layout Generation with a Decoupled Diffusion Model](https://mlsanity.com/paper/2303.05049/) (2023)
- [Relation-Aware Diffusion Model for Controllable Poster Layout Generation](https://mlsanity.com/paper/2306.09086/) (2023)
- [POSTA: A Go-to Framework for Customized Artistic Poster Generation](https://mlsanity.com/paper/2503.14908/) (2025)
- [Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation](https://mlsanity.com/paper/2311.13602/) (2023)
- [PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generation](https://mlsanity.com/paper/2404.00995/) (2024)
- [Geometry Aligned Variational Transformer for Image-conditioned Layout Generation](https://mlsanity.com/paper/2209.00852/) (2022)
- [Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints](https://mlsanity.com/paper/2402.04754/) (2024)
- [LayoutFlow: Flow Matching for Layout Generation](https://mlsanity.com/paper/2403.18187/) (2024)

## Abstract (arXiv)

Content-aware layout generation is a critical task in graphic design automation, focused on creating visually appealing arrangements of elements that seamlessly blend with a given background image. The variety of real-world applications makes it highly challenging to develop a single model capable of unifying the diverse range of input-constrained generation sub-tasks, such as those conditioned by element types, sizes, or their relationships. Current methods either address only a subset of these tasks or necessitate separate model parameters for different conditions, failing to offer a truly unified solution. In this paper, we propose UniLayDiff: a Unified Diffusion Transformer, that for the first time, addresses various content-aware layout generation tasks with a single, end-to-end trainable model. Specifically, we treat layout constraints as a distinct modality and employ Multi-Modal Diffusion Transformer framework to capture the complex interplay between the background image, layout elements, and diverse constraints. Moreover, we integrate relation constraints through fine-tuning the model with LoRA after pretraining the model on other tasks. Such a schema not only achieves unified conditional generation but also enhances overall layout quality. Extensive experiments demonstrate that UniLayDiff achieves state-of-the-art performance across from unconditional to various conditional generation tasks and, to the best of our knowledge, is the first model to unify the full range of content-aware layout generation tasks.