DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Subject-driven image inpainting has recently gained prominence in image editing with the rapid advancement of diffusion models. Beyond image guidance, recent studies have explored incorporating text guidance to achieve identity-preserved yet locally editable object inpainting. However, these methods still suffer from identity overfitting, where original attributes remain entangled with target textual instructions. To overcome this limitation, we propose DreamMix, a diffusion-based framework adept at inserting target objects into user-specified regions while concurrently enabling arbitrary text-driven attribute modifications. DreamMix introduces three key components: (i) an Attribute Decoupling Mechanism (ADM) that synthesizes diverse attribute-augmented image-text pairs to mitigate overfitting; (ii) a Textual Attribute Substitution (TAS) module that isolates target attributes via orthogonal decomposition, and (iii) a Disentangled Inpainting Framework (DIF) that seperates local generation from global harmonization. Extensive experiments across multiple inpainting backbones demonstrate that DreamMix achieves a superior balance between identity preservation and attribute editability across diverse applications, including object insertion, attribute editing, and small object inpainting.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 44 of its 62 references that are in the map. First posted on arXiv 26 Nov 2024.
Builds on
44 of 62 references.
Closest ancestors · 14
Same direction, newest first
- 2024CharacterFactory: Sampling Consistent Characters with GANs for Diffusion Models24 Apr
- 2024Tuning-Free Image Customization with Image and Text Guidance19 Mar
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
- 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Models20 Mar
- 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
- 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
- 2022Paint by Example: Exemplar-based Image Editing with Diffusion Models23 Nov
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
- 2022Blended Latent Diffusion6 Jun
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
2 more
Other references · 30
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
18 more
- 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
- 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
- 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
- 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
- 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
- 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Modelcited by 278
- 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 169
- 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 133
- 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 109
- 2024Zero-shot Image Editing with Reference Imitationcited by 65
- 2024Deep Learning-based Image and Video Inpainting: A Surveycited by 44
- 2023Image-Based Virtual Try-On: A Surveycited by 19
- 2024Selectively Informative Description can Reduce Undesired Embedding Entanglements in Text-to-Image Personalizationcited by 10
- 2023Warping the Residuals for Image Editing with StyleGANcited by 6
- 2024Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Modelcited by 6
- 2024TryOn-Adapter: Efficient Fine-Grained Clothing Identity Adaptation for High-Fidelity Virtual Try-Oncited by 3
- 2022CCR: Facial Image Editing with Continuity, Consistency and Reversibilitycited by 2
Built on by
Citing papers per year: 2025: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2023DreamInpainter: Text-Guided Subject-Driven Image Inpainting with Diffusion Models
- 2024Paint by Inpaint: Learning to Add Image Objects by Removing Them First
- 2023DreamCom: Finetuning Text-guided Inpainting Model for Image Composition
- 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
- 2025InstructAttribute: Fine-grained Object Attributes editing with Instruction
- 2025OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting
- 2026Alterbute: Editing Intrinsic Attributes of Objects in Images
- 2023Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Model