mlsanityData to 14 Oct 2026

DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting

arXiv 2411.17223 ↗First posted 26 Nov 2024cs.CVcited by 2

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Subject-driven image inpainting has recently gained prominence in image editing with the rapid advancement of diffusion models. Beyond image guidance, recent studies have explored incorporating text guidance to achieve identity-preserved yet locally editable object inpainting. However, these methods still suffer from identity overfitting, where original attributes remain entangled with target textual instructions. To overcome this limitation, we propose DreamMix, a diffusion-based framework adept at inserting target objects into user-specified regions while concurrently enabling arbitrary text-driven attribute modifications. DreamMix introduces three key components: (i) an Attribute Decoupling Mechanism (ADM) that synthesizes diverse attribute-augmented image-text pairs to mitigate overfitting; (ii) a Textual Attribute Substitution (TAS) module that isolates target attributes via orthogonal decomposition, and (iii) a Disentangled Inpainting Framework (DIF) that seperates local generation from global harmonization. Extensive experiments across multiple inpainting backbones demonstrate that DreamMix achieves a superior balance between identity preservation and attribute editability across diverse applications, including object insertion, attribute editing, and small object inpainting.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 44 of its 62 references that are in the map. First posted on arXiv 26 Nov 2024.

Builds on

44 of 62 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024CharacterFactory: Sampling Consistent Characters with GANs for Diffusion Models24 Apr
  2. 2024Tuning-Free Image Customization with Image and Text Guidance19 Mar
  3. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  4. 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Models20 Mar
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  6. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  7. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Models23 Nov
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  10. 2022Blended Latent Diffusion6 Jun
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
2 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  2. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
18 more
  1. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  2. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  3. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  4. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
  5. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
  6. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  7. 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Modelcited by 278
  8. 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 169
  9. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 133
  10. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 109
  11. 2024Zero-shot Image Editing with Reference Imitationcited by 65
  12. 2024Deep Learning-based Image and Video Inpainting: A Surveycited by 44
  13. 2023Image-Based Virtual Try-On: A Surveycited by 19
  14. 2024Selectively Informative Description can Reduce Undesired Embedding Entanglements in Text-to-Image Personalizationcited by 10
  15. 2023Warping the Residuals for Image Editing with StyleGANcited by 6
  16. 2024Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Modelcited by 6
  17. 2024TryOn-Adapter: Efficient Fine-Grained Clothing Identity Adaptation for High-Fidelity Virtual Try-Oncited by 3
  18. 2022CCR: Facial Image Editing with Continuity, Consistency and Reversibilitycited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2023DreamInpainter: Text-Guided Subject-Driven Image Inpainting with Diffusion Models
  2. 2024Paint by Inpaint: Learning to Add Image Objects by Removing Them First
  3. 2023DreamCom: Finetuning Text-guided Inpainting Model for Image Composition
  4. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
  5. 2025InstructAttribute: Fine-grained Object Attributes editing with Instruction
  6. 2025OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting
  7. 2026Alterbute: Editing Intrinsic Attributes of Objects in Images
  8. 2023Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Model