mlsanityData to 7 Oct 2026

MagicTailor: Component-Controllable Personalization in Text-to-Image Diffusion Models

arXiv 2410.13370 ↗First posted 17 Oct 2024cs.CV cs.AIcited by 14

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Text-to-image diffusion models can generate high-quality images but lack fine-grained control of visual concepts, limiting their creativity. Thus, we introduce component-controllable personalization, a new task that enables users to customize and reconfigure individual components within concepts. This task faces two challenges: semantic pollution, where undesired elements disrupt the target concept, and semantic imbalance, which causes disproportionate learning of the target concept and component. To address these, we design MagicTailor, a framework that uses Dynamic Masked Degradation to adaptively perturb unwanted visual semantics and Dual-Stream Balancing for more balanced learning of desired visual semantics. The experimental results show that MagicTailor achieves superior performance in this task and enables more personalized and creative image generation.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 57 of its 66 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

57 of 66 references.

Closest ancestors · 22

Same direction, newest first

  1. 2024Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis10 Oct
  2. 2024CSGO: Content-Style Composition in Text-to-Image Generation29 Aug
  3. 2024DiffX: Guide Your Layout to Cross-Modal Generative Modeling22 Jul
  4. 2024A Survey on Personalized Content Synthesis with Diffusion Models9 May
  5. 2024MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation8 Apr
  6. 2024Tuning-Free Image Customization with Image and Text Guidance19 Mar
  7. 2024An Item is Worth a Prompt: Versatile Image Editing with Disentangled Control7 Mar
  8. 2024Diffusion Model-Based Image Editing: A Survey27 Feb
  9. 2024Visual Style Prompting with Swapping Self-Attention20 Feb
  10. 2024DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editing4 Feb
  11. 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
  12. 2023RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths29 May
10 more
  1. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  2. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention17 May
  3. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation30 Mar
  4. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
  5. 2023Cones: Concept Neurons in Diffusion Models for Customized Generation9 Mar
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  7. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  10. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  11. 2017A Closer Look at Memorization in Deep Networkscited by 936
  12. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
23 more
  1. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  2. 2016Generative Adversarial Text to Image Synthesiscited by 735
  3. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  4. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  5. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
  6. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  7. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  8. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  9. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  10. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
  11. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  12. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  13. 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Modelcited by 276
  14. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  15. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  16. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 142
  17. 2020Non-convex Min-Max Optimization: Applications, Challenges, and Recent Theoretical Advancescited by 37
  18. 2023DreamInpainter: Text-Guided Subject-Driven Image Inpainting with Diffusion Modelscited by 30
  19. 2023CLiC: Concept Learning in Contextcited by 20
  20. 2024From Parts to Whole: A Unified Reference Framework for Controllable Human Image Generationcited by 17
  21. 2023Integrating View Conditions for Image Synthesiscited by 16
  22. 2024PartCraft: Crafting Creative Objects by Partscited by 13
  23. 2024Selectively Informative Description can Reduce Undesired Embedding Entanglements in Text-to-Image Personalizationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAscited by 48
  2. 2025JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agentcited by 39
  3. 2024HumanEdit: A High-Quality Human-Rewarded Dataset for Instruction-based Image Editingcited by 29
  4. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  5. 2025WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generationcited by 9
  6. 2024Dual-Schedule Inversion: Training- and Tuning-Free Inversion for Real Image Editingcited by 8
  7. 2025SceneDecorator: Towards Scene-Oriented Story Generation with Scene Planning and Scene Consistencycited by 7
  8. 2026OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generationcited by 7
  9. 2025FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalizationcited by 5
  10. 2026HiFi-Inpaint: Towards High-Fidelity Reference-Based Inpainting for Generating Detail-Preserving Human-Product Imagescited by 5
  11. 2026InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generationcited by 5
  12. 2025Trade-offs in Image Generation: How Do Different Dimensions Interact?cited by 3
2 more
  1. 2025Screen, Cache, and Match: A Training-Free Causality-Consistent Reference Frame Framework for Human Animationcited by 0
  2. 2026ThinkV2V: Unleashing the Reasoning Capability of MLLMs for Instruction-Guided Video Editingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Preserve and Personalize: Personalized Text-to-Image Diffusion Models without Distributional Drift
  2. 2025Semantic Anchoring for Robust Personalization in Text-to-Image Diffusion Models
  3. 2024How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?
  4. 2025ACCORD: Alleviating Concept Coupling through Dependence Regularization for Text-to-Image Diffusion Personalization
  5. 2024Visual Concept-driven Image Generation with Text-to-Image Diffusion Model
  6. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models
  7. 2026LILAC: Layer-Wise Independent LoRAs and Cascaded Conditioning for Multi-Concept Customization of Diffusion Models
  8. 2023Generate Anything Anywhere in Any Scene