mlsanityData to 7 Oct 2026

FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior

arXiv 2407.04947 ↗First posted 6 Jul 2024cs.CVcited by 16

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We offer a novel approach to image composition, which integrates multiple input images into a single, coherent image. Rather than concentrating on specific use cases such as appearance editing (image harmonization) or semantic editing (semantic image composition), we showcase the potential of utilizing the powerful generative prior inherent in large-scale pre-trained diffusion models to accomplish generic image composition applicable to both scenarios. We observe that the pre-trained diffusion models automatically identify simple copy-paste boundary areas as low-density regions during denoising. Building on this insight, we propose to optimize the composed image towards high-density regions guided by the diffusion prior. In addition, we introduce a novel maskguided loss to further enable flexible semantic image composition. Extensive experiments validate the superiority of our approach in achieving generic zero-shot image composition. Additionally, our approach shows promising potential in various tasks, such as object removal and multiconcept customization.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 30 of its 43 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

30 of 43 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Diffusion Model-Based Image Editing: A Survey27 Feb
  2. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Models23 Nov
  3. 2022Blended Latent Diffusion6 Jun
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  6. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov

Other references · 24

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  9. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  10. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  11. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  12. 2017Learning to Discover Cross-Domain Relations with Generative Adversarial Networkscited by 278
12 more
  1. 2023Delta Denoising Scorecited by 120
  2. 2023Noise-Free Score Distillationcited by 99
  3. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 81
  4. 2021SSH: A Self-Supervised Framework for Image Harmonizationcited by 60
  5. 2020Foreground-aware Semantic Representations for Image Harmonizationcited by 53
  6. 2015Learning a Discriminative Model for the Perception of Realism in Composite Imagescited by 52
  7. 2020BargainNet: Background-Guided Domain Translation for Image Harmonizationcited by 52
  8. 2020Transformation Consistency Regularization- A Semi-Supervised Paradigm for Image-to-Image Translationcited by 22
  9. 2023KV Inversion: KV Embeddings Learning for Text-Conditioned Real Image Action Editingcited by 21
  10. 2022DE-Net: Dynamic Text-guided Image Editing Adversarial Networkscited by 12
  11. 2023Deep Image Harmonization in Dual Color Spacescited by 12
  12. 2016On-Demand Learning for Deep Image Restorationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 5.

16 papers.

Sort:

By area:

  1. 2025OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpaintingcited by 38
  2. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 30
  3. 2025Precise Object and Effect Removal with Adaptive Target-Aware Attentioncited by 13
  4. 2025ORIDa: Object-centric Real-world Image Composition Datasetcited by 11
  5. 2025AIComposer: Any Style and Content Image Composition via Feature Integrationcited by 4
  6. 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cuescited by 4
  7. 2025Over++: Generative Video Compositing for Layer Interaction Effectscited by 3
  8. 2025PixPerfect: Seamless Latent Diffusion Local Editing with Discriminative Pixel-Space Refinementcited by 2
  9. 2025CrimEdit: Controllable Editing for Counterfactual Object Removal, Insertion, and Movementcited by 1
  10. 2026DORS: Dynamic Attention Routing for Diffusion-based Object Removal in Dense Scenescited by 1
  11. 2025A Training-Free Approach for Multi-ID Customization via Attention Adjustment and Spatial Controlcited by 0
  12. 2025EraseLoRA: MLLM-Driven Foreground Exclusion and Background Subtype Aggregation for Dataset-Free Object Removalcited by 0
4 more
  1. 2026OmniVTON++: Training-Free Universal Virtual Try-On with Principal Pose Guidancecited by 0
  2. 2026PICS: Pairwise Image Compositing with Spatial Interactionscited by 0
  3. 2026HarmoniDiff-RS: Training-Free Diffusion Harmonization for Satellite Image Compositioncited by 0
  4. 2026Physically Based Rendering in the Latent Spacecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Model
  2. 2023Text-to-Image Diffusion Models are Zero-Shot Classifiers
  3. 2025Structure-Preserving Zero-Shot Image Editing via Stage-Wise Latent Injection in Diffusion Models
  4. 2024RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models
  5. 2024TALE: Training-free Cross-domain Image Composition via Adaptive Latent Manipulation and Energy-guided Optimization
  6. 2023Filter-Guided Diffusion for Controllable Image Generation
  7. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition
  8. 2023Zero-Shot Contrastive Loss for Text-Guided Diffusion Image Style Transfer