mlsanityData to 14 Oct 2026

DreamCom: Finetuning Text-guided Inpainting Model for Image Composition

arXiv 2309.15508 ↗First posted 27 Sep 2023cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

The goal of image composition is merging a foreground object into a background image to obtain a realistic composite image. Recently, generative composition methods are built on large pretrained diffusion models, due to their unprecedented image generation ability. However, they are weak in preserving the foreground object details. Inspired by recent text-to-image generation customized for certain object, we propose DreamCom by treating image composition as text-guided image inpainting customized for certain object. Specifically , we finetune pretrained text-guided image inpainting model based on a few reference images containing the same object, during which the text prompt contains a special token associated with this object. Then, given a new background, we can insert this object into the background with the text prompt containing the special token. In practice, the inserted object may be adversely affected by the background, so we propose masked attention mechanisms to avoid negative background interference. Experimental results on DreamEditBench and our contributed MureCom dataset show the outstanding performance of our DreamCom.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 27 Sep 2023.

Builds on

46 of 57 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  8. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  9. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 695
  10. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  11. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  12. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
34 more
  1. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  2. 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Modelcited by 278
  3. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  4. 2019DoveNet: Deep Image Harmonization via Domain Verificationcited by 135
  5. 2017Deep Image Harmonizationcited by 120
  6. 2018ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositingcited by 90
  7. 2019Improving the Harmony of the Composite Image by Spatial-Separated Attention Modulecited by 73
  8. 2021Region-aware Adaptive Instance Normalization for Image Harmonizationcited by 71
  9. 2018Spatial Fusion GAN for Image Synthesiscited by 68
  10. 2021Making Images Real Again: A Comprehensive Survey on Deep Image Compositioncited by 63
  11. 2019Deep Image Blendingcited by 62
  12. 2022Harmonizer: Learning to Perform White-Box Image and Video Harmonizationcited by 61
  13. 2021SSH: A Self-Supervised Framework for Image Harmonizationcited by 60
  14. 2021High-Resolution Image Harmonization via Collaborative Dual Transformationscited by 55
  15. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Modelscited by 54
  16. 2020Foreground-aware Semantic Representations for Image Harmonizationcited by 53
  17. 2023Putting People in Their Place: Affordance-Aware Human Insertion into Scenescited by 53
  18. 2023Photoswap: Personalized Subject Swapping in Imagescited by 47
  19. 2022DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonizationcited by 46
  20. 2023ControlCom: Controllable Image Composition using Diffusion Modelcited by 43
  21. 2018Context-Aware Synthesis and Placement of Object Instancescited by 41
  22. 2023DreamEdit: Subject-driven Image Editingcited by 37
  23. 2022ObjectStitch: Generative Object Compositingcited by 36
  24. 2021Shadow Generation for Composite Image in Real-world Scenescited by 35
  25. 2020SSN: Soft Shadow Network for Image Compositingcited by 26
  26. 2023Semi-supervised Parametric Real-world Image Harmonizationcited by 26
  27. 2021Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonizationcited by 25
  28. 2022SCS-Co: Self-Consistent Style Contrastive Learning for Image Harmonizationcited by 24
  29. 2020Deep Image Compositingcited by 21
  30. 2022Learning Object Placement via Dual-path Graph Completioncited by 20
  31. 2023LEMaRT: Label-Efficient Masked Region Transform for Image Harmonizationcited by 12
  32. 2021Deep Image-based Illumination Harmonizationcited by 11
  33. 2022Semantic-guided Multi-Mask Image Harmonizationcited by 6
  34. 2023Foreground Object Search by Distilling Composite Image Featurecited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 3; 2026: 4.

17 papers.

Sort:

By area:

  1. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
  2. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  3. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 32
  4. 2024Thinking Outside the BBox: Unconstrained Generative Object Compositingcited by 19
  5. 2024Magic Insert: Style-Aware Drag-and-Dropcited by 17
  6. 2024ObjectMate: A Recurrence Prior for Object Insertion and Subject-Driven Generationcited by 16
  7. 2024Text2Place: Affordance-aware Text Guided Human Placementcited by 11
  8. 2025ORIDa: Object-centric Real-world Image Composition Datasetcited by 11
  9. 2024Affordance-Aware Object Insertion via Mask-Aware Dual Diffusioncited by 6
  10. 2024MureObjectStitch: Multi-reference Image Compositioncited by 4
  11. 2025AIComposer: Any Style and Content Image Composition via Feature Integrationcited by 4
  12. 2024DepGAN: Leveraging Depth Maps for Handling Occlusions and Transparency in Image Compositioncited by 0
5 more
  1. 2024HYPNOS : Highly Precise Foreground-focused Diffusion Finetuning for Inanimate Objectscited by 0
  2. 2026OSInsert: Towards High-authenticity and High-fidelity Image Compositioncited by 0
  3. 2026HiddenObjects: Scalable Diffusion-Distilled Spatial Priors for Object Placementcited by 0
  4. 2026PostureObjectstitch: Anomaly Image Generation Considering Assembly Relationships in Industrial Scenarioscited by 0
  5. 2026BFS: Back-to-Front Layered Image Synthesis via Knowledge Transfercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023DreamInpainter: Text-Guided Subject-Driven Image Inpainting with Diffusion Models
  2. 2024Improving Text-guided Object Inpainting with Semantic Pre-inpainting
  3. 2024I Dream My Painting: Connecting MLLMs and Diffusion Models via Prompt Generation for Text-Guided Multi-Mask Inpainting
  4. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
  5. 2023HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Models
  6. 2024DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting
  7. 2025DreamPainter: Image Background Inpainting for E-commerce Scenarios
  8. 2023Reference-based Image Composition with Sketch via Structure-aware Diffusion Model