mlsanityData to 7 Oct 2026

Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models

arXiv 2305.18292 ↗First posted 29 May 2023cs.CVcited by 261

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Public large-scale text-to-image diffusion models, such as Stable Diffusion, have gained significant attention from the community. These models can be easily customized for new concepts using low-rank adaptations (LoRAs). However, the utilization of multiple concept LoRAs to jointly support multiple customized concepts presents a challenge. We refer to this scenario as decentralized multi-concept customization, which involves single-client concept tuning and center-node concept fusion. In this paper, we propose a new framework called Mix-of-Show that addresses the challenges of decentralized multi-concept customization, including concept conflicts resulting from existing single-client LoRA tuning and identity loss during model fusion. Mix-of-Show adopts an embedding-decomposed LoRA (ED-LoRA) for single-client tuning and gradient fusion for the center node to preserve the in-domain essence of single concepts and support theoretically limitless concept fusion. Additionally, we introduce regionally controllable sampling, which extends spatially controllable sampling (e.g., ControlNet and T2I-Adaptor) to address attribute binding and missing object problems in multi-concept sampling. Extensive experiments demonstrate that Mix-of-Show is capable of composing multiple customized concepts with high fidelity, including characters, objects, and scenes.

Key facts. Cited by 261 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 45 of its 51 references that are in the map. First posted on arXiv 29 May 2023.

Builds on

45 of 51 references.

Closest ancestors · 14

Same direction, newest first

  1. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models5 Apr
  2. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
  3. 2023Cones: Concept Neurons in Diffusion Models for Customized Generation9 Mar
  4. 2023Directed Diffusion: Direct Control of Object Placement through Attention Guidance25 Feb
  5. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  7. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  8. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis9 Dec
  9. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  10. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
2 more
  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  12. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
19 more
  1. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  2. 2020Adaptive Federated Optimizationcited by 1,036
  3. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  6. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  9. 2020Federated Learning with Matched Averagingcited by 564
  10. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  11. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  12. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  13. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  14. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  15. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  16. 2023Anti-DreamBooth: Protecting users from personalized text-to-image synthesiscited by 114
  17. 2022DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latentscited by 103
  18. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  19. 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 45

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 25; 2024: 99; 2025: 90; 2026: 47.

261 papers.

Sort:

By area:

  1. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  2. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  3. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  4. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 280
  5. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  6. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  7. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuningcited by 191
  8. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 186
  9. 2024Mixture of LoRA Expertscited by 179
  10. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 172
  11. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  12. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
38 more
  1. 2024Implicit Style-Content Separation using B-LoRAcited by 135
  2. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  3. 2024Training-Free Consistent Text-to-Image Generationcited by 123
  4. 2023DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Modelscited by 122
  5. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
  6. 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Modelscited by 86
  7. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  8. 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjectscited by 76
  9. 2023Orthogonal Adaptation for Modular Customization of Diffusion Modelscited by 70
  10. 2023VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondencecited by 69
  11. 2024A Survey on LoRA of Large Language Modelscited by 67
  12. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Modelscited by 64
  13. 2024Zero-shot Image Editing with Reference Imitationcited by 64
  14. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 58
  15. 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilitiescited by 58
  16. 2024Any-Resolution AI-Generated Image Detection by Spectral Learningcited by 56
  17. 2024LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Modelscited by 55
  18. 2024I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Modelscited by 55
  19. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Spacecited by 51
  20. 2024MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generationcited by 51
  21. 2025TokenVerse: Versatile Multi-concept Personalization in Token Modulation Spacecited by 51
  22. 2024FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Compositioncited by 49
  23. 2024LoRA Soups: Merging LoRAs for Practical Skill Composition Taskscited by 49
  24. 2025K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAscited by 48
  25. 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalizationcited by 46
  26. 2024MultiBooth: Towards Generating All Your Concepts in an Image from Textcited by 45
  27. 2024Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generationcited by 43
  28. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  29. 2024Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditionscited by 42
  30. 2024DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Controlcited by 42
  31. 2023Paragraph-to-Image Generation with Information-Enriched Diffusion Modelcited by 39
  32. 2023AutoStory: Generating Diverse Storytelling Images with Minimal Human Effortcited by 38
  33. 2024Customizing Text-to-Image Models with a Single Image Paircited by 36
  34. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 35
  35. 2024Low-Rank Adaptation for Foundation Models: A Comprehensive Reviewcited by 35
  36. 2025Efficient Diffusion Models: A Surveycited by 35
  37. 2024Identity Decoupling for Multi-Subject Personalization of Text-to-Image Modelscited by 33
  38. 2024How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?cited by 32

50 most cited of 261 All 261 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025T-LoRA: Single Image Diffusion Model Customization Without Overfitting
  2. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA
  3. 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models
  4. 2025LoRAverse: A Submodular Framework to Retrieve Diverse Adapters for Diffusion Models
  5. 2026CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillation
  6. 2024LoRA Diffusion: Zero-Shot LoRA Synthesis for Diffusion Model Personalization
  7. 2025Blending Concepts with Text-to-Image Diffusion Models
  8. 2026Diffu-LoRA: A Novel Low-Rank Adaptation for Personalized Diffusion Models