# Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generation

arXiv: https://arxiv.org/abs/2409.17920 · HTML: https://mlsanity.com/paper/2409.17920/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 23 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 21 of 21)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation](https://mlsanity.com/paper/2302.13848/) (2023)
- [InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning](https://mlsanity.com/paper/2304.03411/) (2023)
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](https://mlsanity.com/paper/2305.10431/) (2023)
- [Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models](https://mlsanity.com/paper/2305.18292/) (2023)
- [Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs](https://mlsanity.com/paper/2401.11708/) (2024)
- [Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning](https://mlsanity.com/paper/2307.11410/) (2023)
- [Cones: Concept Neurons in Diffusion Models for Customized Generation](https://mlsanity.com/paper/2303.05125/) (2023)
- [SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generation](https://mlsanity.com/paper/2312.16272/) (2023)
- [Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2304.02642/) (2023)
- [MC2: Multi-concept Guidance for Customized Multi-concept Generation](https://mlsanity.com/paper/2404.05268/) (2024)

## Built on by (most cited first, 30 of 43)

- [Less-to-More Generalization: Unlocking More Controllability by In-Context Generation](https://mlsanity.com/paper/2504.02160/) (2025)
- [EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer](https://mlsanity.com/paper/2503.07027/) (2025)
- [XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation](https://mlsanity.com/paper/2506.21416/) (2025)
- [CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design](https://mlsanity.com/paper/2505.19114/) (2025)
- [Personalize Anything for Free with Diffusion Transformer](https://mlsanity.com/paper/2503.12590/) (2025)
- [UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward](https://mlsanity.com/paper/2509.06818/) (2025)
- [PatchDPO: Patch-level DPO for Finetuning-free Personalized Image Generation](https://mlsanity.com/paper/2412.03177/) (2024)
- [ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generation](https://mlsanity.com/paper/2510.11000/) (2025)
- [Query-Kontext: An Unified Multimodal Model for Image Generation and Editing](https://mlsanity.com/paper/2509.26641/) (2025)
- [Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adapter](https://mlsanity.com/paper/2505.18612/) (2025)
- [LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformers](https://mlsanity.com/paper/2505.23758/) (2025)
- [MICo-150K: A Comprehensive Dataset Advancing Multi-Image Composition](https://mlsanity.com/paper/2512.07348/) (2025)
- [Personalized Representation from Personalized Generation](https://mlsanity.com/paper/2412.16156/) (2024)
- [PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewards](https://mlsanity.com/paper/2512.01236/) (2025)
- [MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humans](https://mlsanity.com/paper/2506.20879/) (2025)
- [AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models](https://mlsanity.com/paper/2412.04146/) (2024)
- [LAMIC: Layout-Aware Multi-Image Composition via Scalability of Multimodal Diffusion Transformer](https://mlsanity.com/paper/2508.00477/) (2025)
- [ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generation](https://mlsanity.com/paper/2509.18092/) (2025)
- [Controllable Human Image Generation with Personalized Multi-Garments](https://mlsanity.com/paper/2411.16801/) (2024)
- [FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization](https://mlsanity.com/paper/2507.01792/) (2025)
- [FreeCond: Free Lunch in the Input Conditions of Text-Guided Inpainting](https://mlsanity.com/paper/2412.00427/) (2024)
- [TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models](https://mlsanity.com/paper/2601.08011/) (2026)
- [Dialogue Director: Bridging the Gap in Dialogue Visualization for Multimodal Storytelling](https://mlsanity.com/paper/2412.20725/) (2024)
- [Interact-Custom: Customized Human Object Interaction Image Generation](https://mlsanity.com/paper/2508.19575/) (2025)
- [FocusDPO: Dynamic Preference Optimization for Multi-Subject Personalized Image Generation via Adaptive Focus](https://mlsanity.com/paper/2509.01181/) (2025)
- [ChangeBridge: Spatiotemporal Image Generation with Multimodal Controls for Remote Sensing](https://mlsanity.com/paper/2507.04678/) (2025)
- [ConceptWeaver: Weaving Disentangled Concepts with Flow](https://mlsanity.com/paper/2603.28493/) (2026)
- [Multimodal Large Language Models for Multi-Subject In-Context Image Generation](https://mlsanity.com/paper/2604.07422/) (2026)
- [VMDiff: Visual Mixing Diffusion for Limitless Cross-Object Synthesis](https://mlsanity.com/paper/2509.23605/) (2025)
- [Adversarial Concept Distillation for One-Step Diffusion Personalization](https://mlsanity.com/paper/2510.20512/) (2025)

## Abstract (arXiv)

Personalized text-to-image generation methods can generate customized images based on the reference images, which have garnered wide research interest. Recent methods propose a finetuning-free approach with a decoupled cross-attention mechanism to generate personalized images requiring no test-time finetuning. However, when multiple reference images are provided, the current decoupled cross-attention mechanism encounters the object confusion problem and fails to map each reference image to its corresponding object, thereby seriously limiting its scope of application. To address the object confusion problem, in this work we investigate the relevance of different positions of the latent image features to the target object in diffusion model, and accordingly propose a weighted-merge method to merge multiple reference image features into the corresponding objects. Next, we integrate this weighted-merge method into existing pre-trained models and continue to train the model on a multi-object dataset constructed from the open-sourced SA-1B dataset. To mitigate object confusion and reduce training costs, we propose an object quality score to estimate the image quality for the selection of high-quality training samples. Furthermore, our weighted-merge training framework can be employed on single-object generation when a single object has multiple reference images. The experiments verify that our method achieves superior performance to the state-of-the-arts on the Concept101 dataset and DreamBooth dataset of multi-object personalized image generation, and remarkably improves the performance on single-object personalized image generation. Our code is available at https://github.com/hqhQAQ/MIP-Adapter.