Mise-en-Sc`ene: Implicit Layout Emergence in Diffusion Transformers for Human-AI Design Co-Creation
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Automating graphic design synthesis from user-provided elements requires both a coherent overall composition and the exact preservation of each asset. Existing methods predict a layout as explicit bounding-box coordinates with a language model and then paste the assets into it, which separates spatial planning from visual synthesis and tends to produce rigid, mis-scaled compositions. We instead ask whether the layout can emerge implicitly inside a pretrained image-editing diffusion transformer. We present Mise-en-Sc`ene, a two-stage framework. In the first stage, a diffusion transformer adapted with a small, knockout-selected LoRA drafts a complete design in which the arrangement of the elements emerges jointly with the rendered canvas. In the second stage, a deterministic match-and-place step moves the original high-resolution assets to the drafted positions, which guarantees exact asset fidelity and yields an editable, layered design that a designer can keep refining rather than a flat image. Notably, a minimal adaptation of the pretrained transformer already suffices, without the specialized conditioning machinery commonly introduced for multi-element generation. On the large-scale PrismLayersPlus benchmark, the designs produced by Mise-en-Sc`ene are the closest to the ground truth in perceived quality among all compared methods, by a wide margin over both an LLM layout planner and a specialized layout transformer, while our match-and-place stage bridges the remaining fidelity gap to the ground-truth composites.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 22 of its 53 references that are in the map. First posted on arXiv 19 Aug 2026.
Builds on
22 of 53 references.
Closest ancestors · 4
Same direction, newest first
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space17 Jun
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design25 May
- 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning10 Apr
- 2025OminiControl2: Efficient Conditioning for Diffusion Transformers11 Mar
Other references · 18
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2025Qwen-Image Technical Reportcited by 1,044
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
- 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 154
- 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 147
- 2019LayoutGAN: Generating Graphic Layouts with Wireframe Discriminatorscited by 131
- 2023COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Designcited by 43
- 2024PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLMcited by 41
- 2025PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Frameworkcited by 40
- 2025ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generationcited by 37
- 2025Qwen-Image-Layered: Towards Inherent Editability via Layer Decompositioncited by 34
- 2025LayerD: Decomposing Raster Graphic Designs into Layerscited by 21
6 more
- 2025BizGen: Advancing Article-level Visual Text Rendering for Infographics Generationcited by 20
- 2025CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic Design Generationcited by 19
- 2024Multimodal Markup Document Models for Graphic Design Completioncited by 16
- 2025PrismLayers: Open Data for High-Quality Multi-Layer Transparent Image Generative Modelscited by 15
- 2025IP-Composer: Semantic Composition of Visual Conceptscited by 14
- 2025Piece it Together: Part-Based Concepting with IP-Priorscited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation
- 2024Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints
- 2024LayoutDiT: Exploring Content-Graphic Balance in Layout Generation with Diffusion Transformer
- 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generation
- 2025Laytrol: Preserving Pretrained Knowledge in Layout Control for Multimodal Diffusion Transformers
- 2025LayoutAgent: A Vision-Language Agent Guided Compositional Diffusion for Spatial Layout Planning
- 2023Conditional Human Sketch Synthesis with Explicit Abstraction Control
- 2026Appearance Pointers -- Multimodal Region Control of Diffusion Transformers