mlsanityData to 14 Oct 2026

CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design

arXiv 2505.19114 ↗First posted 25 May 2025cs.CVcited by 36

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Graphic design plays a vital role in visual communication across advertising, marketing, and multimedia entertainment. Prior work has explored automated graphic design generation using diffusion models, aiming to streamline creative workflows and democratize design capabilities. However, complex graphic design scenarios require accurately adhering to design intent specified by multiple heterogeneous user-provided elements (eg images, layouts, and texts), which pose multi-condition control challenges for existing methods. Specifically, previous single-condition control models demonstrate effectiveness only within their specialized domains but fail to generalize to other conditions, while existing multi-condition methods often lack fine-grained control over each sub-condition and compromise overall compositional harmony. To address these limitations, we introduce CreatiDesign, a systematic solution for automated graphic design covering both model architecture and dataset construction. First, we design a unified multi-condition driven architecture that enables flexible and precise integration of heterogeneous design elements with minimal architectural modifications to the base diffusion model. Furthermore, to ensure that each condition precisely controls its designated image region and to avoid interference between conditions, we propose a multimodal attention mask mechanism. Additionally, we develop a fully automated pipeline for constructing graphic design datasets, and introduce a new dataset with 400K samples featuring multi-condition annotations, along with a comprehensive benchmark. Experimental results show that CreatiDesign outperforms existing models by a clear margin in faithfully adhering to user intent.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 105 of its 119 references that are in the map. First posted on arXiv 25 May 2025.

Builds on

105 of 119 references.

Closest ancestors · 48

Same direction, newest first

  1. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer29 Apr
  2. 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning10 Apr
  3. 2025Multi-party Collaborative Attention Control for Image Customization2 Apr
  4. 2025DynASyn: Multi-Subject Personalization Enabling Dynamic Action Synthesis22 Mar
  5. 2025Personalize Anything for Free with Diffusion Transformer16 Mar
  6. 2025UniCombine: Unified Multi-Conditional Combination with Diffusion Transformer12 Mar
  7. 2025Seedream 2.0: A Native Chinese-English Bilingual Image Generation Foundation Model10 Mar
  8. 2025PixelPonder: Dynamic Patch Adaptation for Enhanced Multi-Conditional Text-to-Image Generation9 Mar
  9. 2025DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models3 Mar
  10. 2025DC-ControlNet: Decoupling Inter- and Intra-Element Conditions in Image Generation with Diffusion Models20 Feb
  11. 2025One-Prompt-One-Story: Free-Lunch Consistent Text-to-Image Generation Using a Single Prompt23 Jan
  12. 2025EliGen: Entity-Level Controlled Image Generation with Regional Attention2 Jan
36 more
  1. 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generation5 Dec
  2. 2024Diffusion Self-Distillation for Zero-Shot Customized Image Generation27 Nov
  3. 2024Training-free Regional Prompting for Diffusion Transformers4 Nov
  4. 2024In-Context LoRA for Diffusion Transformers31 Oct
  5. 2024GrounDiT: Grounding Diffusion Transformers via Noisy Patch Transplantation27 Oct
  6. 2024Group Diffusion Transformers are Unsupervised Multitask Learners19 Oct
  7. 2024HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generation18 Oct
  8. 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformer30 Sep
  9. 2024OmniGen: Unified Image Generation17 Sep
  10. 2024IFAdapter: Instance Feature Control for Grounded Text-to-Image Generation12 Sep
  11. 2024JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation8 Jul
  12. 2024GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models2 Jul
  13. 2024AnyControl: Create Your Artwork with Versatile Control on Text-to-Image Generation27 Jun
  14. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding14 May
  15. 2024MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation8 Apr
  16. 2024Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generation25 Mar
  17. 2024NoiseCollage: A Layout-Aware Text-to-Image Diffusion Model Based on Noise Cropping and Merging6 Mar
  18. 2024InstanceDiffusion: Instance-level Control for Image Generation5 Feb
  19. 2023X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model4 Dec
  20. 2023Reason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis28 Nov
  21. 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following28 Nov
  22. 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering28 Nov
  23. 2023Check, Locate, Rectify: A Training-Free Layout Calibration System for Text-to-Image Generation27 Nov
  24. 2023SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generation20 Aug
  25. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  26. 2023LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts13 Aug
  27. 2023Grounded Text-to-Image Synthesis with Attention Refocusing8 Jun
  28. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
  29. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild18 May
  30. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation30 Mar
  31. 2023Freestyle Layout-to-Image Synthesis25 Mar
  32. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generation14 Mar
  33. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  34. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  35. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  36. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May

Other references · 57

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2022Scalable Diffusion Models with Transformerscited by 6,054
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  12. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
38 more
  1. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  2. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  3. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  4. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  5. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
  6. 2019PubLayNet: largest dataset ever for document layout analysiscited by 351
  7. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  8. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 154
  9. 2025Seedream 3.0 Technical Reportcited by 154
  10. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
  11. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  12. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 124
  13. 2013Stabilitycited by 120
  14. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
  15. 2024Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Modelscited by 116
  16. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 114
  17. 2025DreamO: A Unified Framework for Image Customizationcited by 83
  18. 2024Glyph-ByT5: A Customized Text Encoder for Accurate Visual Text Renderingcited by 81
  19. 2024Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Modelcited by 66
  20. 2024Instruct-Imagen: Image Generation with Multi-modal Instructioncited by 65
  21. 2024AnyText2: Visual Text Generation and Editing With Customizable Attributescited by 53
  22. 2024FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Compositioncited by 49
  23. 2023LayoutPrompter: Awaken the Design Ability of Large Language Modelscited by 48
  24. 2024Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generatorcited by 48
  25. 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalizationcited by 47
  26. 2025PosterMaker: Towards High-Quality Product Poster Generation with Accurate Text Renderingcited by 46
  27. 2024MultiBooth: Towards Generating All Your Concepts in an Image from Textcited by 45
  28. 2023COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Designcited by 43
  29. 2024Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generationcited by 43
  30. 2023AutoPoster: A Highly Automatic and Content-aware Design System for Advertising Poster Generationcited by 41
  31. 2024One Diffusion to Generate Them Allcited by 41
  32. 2024Graphic Design with Large Multimodal Modelcited by 40
  33. 2024RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customizationcited by 39
  34. 2025ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generationcited by 37
  35. 2025POSTA: A Go-to Framework for Customized Artistic Poster Generationcited by 36
  36. 2023Towards Flexible Multi-modal Document Modelscited by 31
  37. 2025DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale Text-to-Image Modelscited by 28
  38. 2025Generating Multi-Image Synthetic Data for Text-to-Image Customizationcited by 25

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 26.

36 papers.

Sort:

By area:

  1. 2025DreamPoster: A Unified Framework for Image-Conditioned Generative Poster Designcited by 16
  2. 2025PosterGen: Aesthetic-Aware Multi-Modal Paper-to-Poster Generation via Multi-Agent LLMscited by 11
  3. 2025Seg2Any: Open-set Segmentation-Mask-to-Image Generation with Precise Shape and Semantic Controlcited by 10
  4. 2026PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedbackcited by 9
  5. 2026CreatiParser: Generative Image Parsing of Raster Graphic Designs into Editable Layerscited by 9
  6. 2025EmoVerse: A MLLMs-Driven Emotion Representation Dataset for Interpretable Visual Emotion Analysiscited by 7
  7. 2026WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editingcited by 7
  8. 2025PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Designcited by 6
  9. 2026InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generationcited by 6
  10. 2025TerraGen: A Unified Multi-Task Layout Generation Framework for Remote Sensing Data Augmentationcited by 5
  11. 2025Canvas-to-Image: Compositional Image Generation with Multimodal Controlscited by 5
  12. 2026DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generationcited by 5
24 more
  1. 2026Layer-wise Instance Binding for Regional and Occlusion Control in Text-to-Image Diffusion Transformerscited by 5
  2. 2026RefineAnything: Multimodal Region-Specific Refinement for Perfect Local Detailscited by 5
  3. 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
  4. 2025PositionIC: Unified Position and Identity Consistency for Image Customizationcited by 4
  5. 2025CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvascited by 3
  6. 2026Audio-Visual Exchange-Aware Token Pruning for Efficient Audio-Visual Captioningcited by 3
  7. 2026EmoStyle: Affective Conditioning of Style-Specialist Experts for Emotional Image Generationcited by 3
  8. 2026PSDesigner: Automated Graphic Design with a Human-Like Creative Workflowcited by 2
  9. 2025Paper2Web: Let's Make Your Paper Alive!cited by 1
  10. 2026AnyDoc: Enhancing Document Generation via Large-Scale HTML/CSS Data Synthesis and Height-Aware Reinforcement Optimizationcited by 1
  11. 2026Does Synthetic Layered Design Data Benefit Layered Design Decomposition?cited by 1
  12. 2026Geometry-aware Gaussian Prior and Axial Attention for Cervical Cytology Image Classificationcited by 1
  13. 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editingcited by 1
  14. 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 1
  15. 2026Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Designcited by 1
  16. 2026Trajectory-Guided Diffusion for Foreground-Preserving Background Generation in Multi-Layer Documentscited by 0
  17. 2026Towards Design Compositingcited by 0
  18. 2026Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generationcited by 0
  19. 2026Through the PRISM: Principle-Aware, Interpretable, and Multi-Scale Evaluation of Visual Designscited by 0
  20. 2026InstanceControl: Controllable Complex Image Generation without Instance Labelingcited by 0
  21. 2026Rationale-Guided Knowledge Distillation for Cross-Lingual Stance Detectioncited by 0
  22. 2026Mise-en-Sc`ene: Implicit Layout Emergence in Diffusion Transformers for Human-AI Design Co-Creationcited by 0
  23. 2026Learning Interaction between Image and Layout Priors for Joint Image-Layout Generation in Design Templatescited by 0
  24. 2026Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation
  2. 2024Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints
  3. 2024LayoutDiT: Exploring Content-Graphic Balance in Layout Generation with Diffusion Transformer
  4. 2025CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic Design Generation
  5. 2024DiffDesign: Controllable Diffusion with Meta Prior for Efficient Interior Design Generation
  6. 2026iPoster: Content-Aware Layout Generation for Interactive Poster Design via Graph-Enhanced Diffusion Models
  7. 2026Appearance Pointers -- Multimodal Region Control of Diffusion Transformers
  8. 2025IGD: Instructional Graphic Design with Multimodal Layer Generation