mlsanityData to 14 Oct 2026

MUSE: Multi-Subject Unified Synthesis via Explicit Layout Semantic Expansion

arXiv 2508.14440 ↗First posted 20 Aug 2025cs.CVcited by 4

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Existing text-to-image diffusion models have demonstrated remarkable capabilities in generating high-quality images guided by textual prompts. However, achieving multi-subject compositional synthesis with precise spatial control remains a significant challenge. In this work, we address the task of layout-controllable multi-subject synthesis (LMS), which requires both faithful reconstruction of reference subjects and their accurate placement in specified regions within a unified image. While recent advancements have separately improved layout control and subject synthesis, existing approaches struggle to simultaneously satisfy the dual requirements of spatial precision and identity preservation in this composite task. To bridge this gap, we propose MUSE, a unified synthesis framework that employs concatenated cross-attention (CCA) to seamlessly integrate layout specifications with textual guidance through explicit semantic space expansion. The proposed CCA mechanism enables bidirectional modality alignment between spatial constraints and textual descriptions without interference. Furthermore, we design a progressive two-stage training strategy that decomposes the LMS task into learnable sub-objectives for effective optimization. Extensive experiments demonstrate that MUSE achieves zero-shot end-to-end generation with superior spatial accuracy and identity consistency compared to existing solutions, advancing the frontier of controllable image synthesis. Our code and model are available at https://github.com/pf0607/MUSE.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 61 of its 68 references that are in the map. First posted on arXiv 20 Aug 2025.

Builds on

61 of 68 references.

Closest ancestors · 26

Same direction, newest first

  1. 2024IFAdapter: Instance Feature Control for Grounded Text-to-Image Generation12 Sep
  2. 2024Move Anything with Layered Scene Diffusion10 Apr
  3. 2024NoiseCollage: A Layout-Aware Text-to-Image Diffusion Model Based on Noise Cropping and Merging6 Mar
  4. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space7 Feb
  5. 2024InstanceDiffusion: Instance-level Control for Image Generation5 Feb
  6. 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
  7. 2023Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion5 Oct
  8. 2023Dense Text-to-Image Generation with Attention Modulation24 Aug
  9. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  10. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning21 Jul
  11. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion20 Jul
  12. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuning12 Jun
14 more
  1. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  2. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
  3. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation16 Feb
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  6. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  7. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis9 Dec
  8. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  9. 2022SpaText: Spatio-Textual Representation for Controllable Image Generation25 Nov
  10. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  13. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  14. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2022Generative Adversarial Networkscited by 5,627
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
23 more
  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  2. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  4. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  5. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  6. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
  7. 2016Generative Adversarial Text to Image Synthesiscited by 735
  8. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  9. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  10. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  11. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
  12. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  13. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
  14. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  15. 2023Recognize Anything: A Strong Image Tagging Modelcited by 309
  16. 2023Generating Images with Multimodal Language Modelscited by 260
  17. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 179
  18. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
  19. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
  20. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 124
  21. 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Modelscited by 105
  22. 2024RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customizationcited by 39
  23. 2024MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesiscited by 35

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Obtaining Favorable Layouts for Multiple Object Generation
  2. 2025HCMA: Hierarchical Cross-model Alignment for Grounded Text-to-Image Generation
  3. 2023LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation
  4. 2023LoCo: Locally Constrained Training-Free Layout-to-Image Synthesis
  5. 2024Training-free Composite Scene Generation for Layout-to-Image Synthesis
  6. 2025A Two-Stage System for Layout-Controlled Image Generation using Large Language Models and Diffusion Models
  7. 2026All-in-One Conditioning for Text-to-Image Synthesis
  8. 2024Layout-to-Image Generation with Localized Descriptions using ControlNet with Cross-Attention Control