MS-CustomNet: Controllable Multi-Subject Customization with Hierarchical Relational Semantics
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Diffusion-based text-to-image generation has advanced significantly, yet customizing scenes with multiple distinct subjects while maintaining fine-grained control over their interactions remains challenging. Existing methods often struggle to provide explicit user-defined control over the compositional structure and precise spatial relationships between subjects. To address this, we introduce MS-CustomNet, a novel framework for multi-subject customization. MS-CustomNet allows zero-shot integration of multiple user-provided objects and, crucially, empowers users to explicitly define these hierarchical arrangements and spatial placements within the generated image. Our approach ensures individual subject identity preservation while learning and enacting these user-specified inter-subject compositions. We also present the MSI dataset, derived from COCO, to facilitate training on such complex multi-subject compositions. MS-CustomNet offers enhanced, fine-grained control over multi-subject image generation. Our method achieves a DINO-I score of 0.61 for identity preservation and a YOLO-L score of 0.94 for positional control in multi-subject customization tasks, demonstrating its superior capability in generating high-fidelity images with precise, user-directed multi-subject compositions and spatial control.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 38 of its 44 references that are in the map. First posted on arXiv 22 Mar 2026.
Builds on
38 of 44 references.
Closest ancestors · 14
Same direction, newest first
- 2024How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?23 Oct
- 2024JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation8 Jul
- 2024LoMOE: Localized Multi-Object Editing via Multi-Diffusion1 Mar
- 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space7 Feb
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs22 Jan
- 2023Orthogonal Adaptation for Modular Customization of Diffusion Models5 Dec
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
- 2023Cones: Concept Neurons in Diffusion Models for Customized Generation9 Mar
- 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
- 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
- 2022Paint by Example: Exemplar-based Image Editing with Diffusion Models23 Nov
2 more
Other references · 24
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
12 more
- 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
- 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
- 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 414
- 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
- 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
- 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 186
- 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 142
- 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
- 2023DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generationcited by 75
- 2024FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Compositioncited by 49
- 2024GALA: Graph Diffusion-based Alignment with Jigsaw for Source-free Domain Adaptationcited by 8
- 2024Diff-DTI: Fast Diffusion Tensor Imaging Using A Feature-Enhanced Joint Diffusion Modelcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023CustomNet: Zero-shot Object Customization with Variable-Viewpoints in Text-to-Image Diffusion Models
- 2025Multi-party Collaborative Attention Control for Image Customization
- 2025XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation
- 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning
- 2026Redirecting the Flow: Image Customization through Attention Distribution Shift
- 2025MVCustom: Multi-View Customized Diffusion via Geometric Latent Rendering and Completion
- 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjects
- 2026MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding