mlsanityData to 7 Oct 2026

Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models

arXiv 2404.03913 ↗First posted 5 Apr 2024cs.CV cs.AI cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

While there has been significant progress in customizing text-to-image generation models, generating images that combine multiple personalized concepts remains challenging. In this work, we introduce Concept Weaver, a method for composing customized text-to-image diffusion models at inference time. Specifically, the method breaks the process into two steps: creating a template image aligned with the semantics of input prompts, and then personalizing the template using a concept fusion strategy. The fusion strategy incorporates the appearance of the target concepts into the template image while retaining its structural details. The results indicate that our method can generate multiple custom concepts with higher identity fidelity compared to alternative approaches. Furthermore, the method is shown to seamlessly handle more than two concepts and closely follow the semantic meaning of the input prompt without blending appearances across different subjects.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 28 of its 30 references that are in the map. First posted on arXiv 5 Apr 2024.

Builds on

28 of 30 references.

References · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2023Segment Anythingcited by 8,158
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
16 more
  1. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  2. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  3. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  4. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  5. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  6. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  7. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  8. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  9. 2022Inversion-Based Style Transfer with Diffusion Modelscited by 290
  10. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  11. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wildcited by 197
  12. 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
  13. 2023Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningcited by 168
  14. 2023Improving Tuning-Free Real Image Editing with Proximal Guidancecited by 87
  15. 2023Energy-Based Cross Attention for Bayesian Context Update in Text-to-Image Diffusion Modelscited by 30
  16. 2023Generate Anything Anywhere in Any Scenecited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 15; 2026: 3.

23 papers.

Sort:

By area:

  1. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 153
  2. 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalizationcited by 46
  3. 2025Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Promptscited by 28
  4. 2025Personalize Anything for Free with Diffusion Transformercited by 22
  5. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  6. 2024ClassDiffusion: More Aligned Personalization Tuning with Explicit Class Guidancecited by 18
  7. 2025AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generationcited by 16
  8. 2024TweedieMix: Improving Multi-Concept Fusion for Diffusion-based Image/Video Generationcited by 14
  9. 2025LatexBlend: Scaling Multi-concept Customized Generation with Latent Textual Blendingcited by 10
  10. 2025MultLFG: Training-free Multi-LoRA composition using Frequency-domain Guidancecited by 6
  11. 2024Concept Conductor: Orchestrating Multiple Personalized Concepts in Text-to-Image Synthesiscited by 4
  12. 2025Continual Personalization for Diffusion Modelscited by 3
11 more
  1. 2025FlipConcept: Tuning-Free Multi-Concept Personalization for Text-to-Image Generationcited by 2
  2. 2025Composing Concepts from Images and Videos via Concept-prompt Bindingcited by 2
  3. 2024SPDiffusion: Semantic Protection Diffusion Models for Multi-concept Text-to-image Generationcited by 1
  4. 2025JEDI: The Force of Jensen-Shannon Divergence in Disentangling Diffusion Modelscited by 1
  5. 2025AnyMS: Bottom-up Attention Decoupling for Layout-guided and Training-free Multi-subject Customizationcited by 1
  6. 2025FaR: Enhancing Multi-Concept Text-to-Image Diffusion via Concept Fusion and Localized Refinementcited by 0
  7. 2025FreeFuse: Multi-Subject LoRA Fusion via Adaptive Token-Level Routing at Test Timecited by 0
  8. 2025A Training-Free Approach for Multi-ID Customization via Attention Adjustment and Spatial Controlcited by 0
  9. 2026Training-Free Multi-Concept LoRA Composition with Prompt-Aware Weightingcited by 0
  10. 2026MultiAnimate: A Unified Framework for Controllable Multi-Character Animationcited by 0
  11. 2026X-MULTI: VLM-based Imaging Factor Disentanglement for Factor-Aware Image Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Visual Concept-driven Image Generation with Text-to-Image Diffusion Model
  2. 2023Multi-Concept T2I-Zero: Tweaking Only The Text Embeddings and Nothing Else
  3. 2024Gen4Gen: Generative Data Pipeline for Generative Multi-Concept Composition
  4. 2024Fast Personalized Text-to-Image Syntheses With Attention Injection
  5. 2025Blending Concepts with Text-to-Image Diffusion Models
  6. 2024Non-confusing Generation of Customized Concepts in Diffusion Models
  7. 2026Injecting Image Guidance into Text-Conditioned Diffusion Models at Inference
  8. 2025Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adapter