mlsanityData to 7 Oct 2026

MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidance

arXiv 2406.07209 ↗First posted 11 Jun 2024cs.CVcited by 142

Not on the map. Why?

Cited by 44 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advancements in text-to-image generation models have dramatically enhanced the generation of photorealistic images from textual prompts, leading to an increased interest in personalized text-to-image applications, particularly in multi-subject scenarios. However, these advances are hindered by two main challenges: firstly, the need to accurately maintain the details of each referenced subject in accordance with the textual descriptions; and secondly, the difficulty in achieving a cohesive representation of multiple subjects in a single image without introducing inconsistencies. To address these concerns, our research introduces the MS-Diffusion framework for layout-guided zero-shot image personalization with multi-subjects. This innovative approach integrates grounding tokens with the feature resampler to maintain detail fidelity among subjects. With the layout guidance, MS-Diffusion further improves the cross-attention to adapt to the multi-subject inputs, ensuring that each subject condition acts on specific areas. The proposed multi-subject cross-attention orchestrates harmonious inter-subject compositions while preserving the control of texts. Comprehensive quantitative and qualitative experiments affirm that this method surpasses existing models in both image and text fidelity, promoting the development of personalized text-to-image generation. The project page is https://MS-Diffusion.github.io.

Key facts. Cited by 142 papers in the map (data through 7 Oct 2026). Builds on 51 of its 57 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

51 of 57 references.

References · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023Segment Anythingcited by 8,158
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
38 more
  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  8. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  10. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  11. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  12. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  13. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  14. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  15. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  16. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  17. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  18. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  19. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  20. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  21. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  22. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  23. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  24. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
  25. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 233
  26. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  27. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuningcited by 207
  28. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuningcited by 191
  29. 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
  30. 2023Dense Text-to-Image Generation with Attention Modulationcited by 162
  31. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 142
  32. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
  33. 2023Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusioncited by 114
  34. 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Modelscited by 105
  35. 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101
  36. 2023DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generationcited by 75
  37. 2024Instruct-Imagen: Image Generation with Multi-modal Instructioncited by 65
  38. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Spacecited by 51

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 76; 2026: 44.

142 papers.

Sort:

By area:

  1. 2024OmniGen: Unified Image Generationcited by 388
  2. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 153
  3. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
  4. 2024CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Modelscited by 99
  5. 2025DreamO: A Unified Framework for Image Customizationcited by 82
  6. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  7. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 67
  8. 2025XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulationcited by 55
  9. 2024Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generatorcited by 48
  10. 2024Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinementcited by 37
  11. 2025ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generationcited by 37
  12. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
38 more
  1. 2024Training-free Regional Prompting for Diffusion Transformerscited by 26
  2. 2025Personalize Anything for Free with Diffusion Transformercited by 22
  3. 2025MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglementcited by 22
  4. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  5. 2024Personalized Multimodal Large Language Models: A Surveycited by 19
  6. 2025CINEMA: Coherent Multi-Subject Video Generation via MLLM-Based Guidancecited by 19
  7. 2024PatchDPO: Patch-level DPO for Finetuning-free Personalized Image Generationcited by 18
  8. 2025UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Rewardcited by 18
  9. 2025ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generationcited by 17
  10. 2024RealCustom++: Representing Images as Real Textual Word for Real-Time Customizationcited by 16
  11. 2024DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generationcited by 16
  12. 2025RealGeneral: Unifying Visual Generation via Temporal In-Context Learning with Video Modelscited by 16
  13. 2025MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Modelscited by 16
  14. 2025OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Modelscited by 16
  15. 2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editingcited by 15
  16. 2024CustomContrast: A Multilevel Contrastive Perspective For Subject-Driven Text-to-Image Customizationcited by 13
  17. 2024GroundingBooth: Grounding Text-to-Image Customizationcited by 13
  18. 2025Unified Dense Prediction of Video Diffusioncited by 13
  19. 2025Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adaptercited by 13
  20. 2025MOSAIC: Multi-Subject Personalized Generation via Correspondence-Aware Alignment and Disentanglementcited by 13
  21. 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memorycited by 13
  22. 2024T2I-FactualBench: Benchmarking the Factuality of Text-to-Image Models with Knowledge-Intensive Conceptscited by 12
  23. 2025LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformerscited by 12
  24. 2025CustomVideoX: 3D Reference Attention Driven Dynamic Adaptation for Zero-Shot Customized Video Diffusion Transformerscited by 11
  25. 2025Flux Already Knows -- Activating Subject-Driven Image Generation without Trainingcited by 11
  26. 2025MUSAR: Exploring Multi-Subject Customization from Single-Subject Dataset via Attention Routingcited by 11
  27. 2025OmniPSD: Layered PSD Generation with Diffusion Transformercited by 11
  28. 2025LatexBlend: Scaling Multi-concept Customized Generation with Latent Textual Blendingcited by 10
  29. 2025Be Decisive: Noise-Induced Layouts for Multi-Subject Generationcited by 10
  30. 2024DreamRelation: Bridging Customization and Relation Generationcited by 9
  31. 2025First Frame Is the Place to Go for Video Content Customizationcited by 9
  32. 2025MICo-150K: A Comprehensive Dataset Advancing Multi-Image Compositioncited by 9
  33. 2025MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humanscited by 8
  34. 2025MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generationcited by 8
  35. 2025MoEdit: On Learning Quantity Perception for Multi-object Image Editingcited by 7
  36. 2025LAMIC: Layout-Aware Multi-Image Composition via Scalability of Multimodal Diffusion Transformercited by 7
  37. 2025ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generationcited by 7
  38. 2025PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewardscited by 7

50 most cited of 142 All 142 (JSON)