mlsanityData to 14 Oct 2026

TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition

arXiv 2307.12493 ↗First posted 24 Jul 2023cs.CV cs.AIcited by 182

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Text-driven diffusion models have exhibited impressive generative capabilities, enabling various image editing tasks. In this paper, we propose TF-ICON, a novel Training-Free Image COmpositioN framework that harnesses the power of text-driven diffusion models for cross-domain image-guided composition. This task aims to seamlessly integrate user-provided objects into a specific visual context. Current diffusion-based methods often involve costly instance-based optimization or finetuning of pretrained models on customized datasets, which can potentially undermine their rich prior. In contrast, TF-ICON can leverage off-the-shelf diffusion models to perform cross-domain image-guided composition without requiring additional training, finetuning, or optimization. Moreover, we introduce the exceptional prompt, which contains no information, to facilitate text-driven diffusion models in accurately inverting real images into latent representations, forming the basis for compositing. Our experiments show that equipping Stable Diffusion with the exceptional prompt outperforms state-of-the-art inversion methods on various datasets (CelebA-HQ, COCO, and ImageNet), and that TF-ICON surpasses prior baselines in versatile visual domains. Code is available at https://github.com/Shilin-LU/TF-ICON

Key facts. Cited by 182 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 77 of its 90 references that are in the map. First posted on arXiv 24 Jul 2023.

Builds on

77 of 90 references.

Closest ancestors · 17

Same direction, newest first

  1. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  2. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  3. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis9 Dec
  4. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  5. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Models23 Nov
  6. 2022Investigating Prompt Engineering in Diffusion Models21 Nov
  7. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  8. 2022DiffEdit: Diffusion-based semantic image editing with mask guidance20 Oct
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  11. 2022Blended Latent Diffusion6 Jun
  12. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun
5 more
  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  4. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov
  5. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 60

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  12. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
38 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  3. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  4. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  5. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  6. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  7. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  8. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  9. 2021Variational Diffusion Modelscited by 1,153
  10. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  11. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  12. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  13. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  14. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 700
  15. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  16. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  17. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  18. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  19. 2023Zero-shot Image-to-Image Translationcited by 505
  20. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  21. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  22. 2022Diffusion Models already have a Semantic Latent Spacecited by 328
  23. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 307
  24. 2022Perception Prioritized Training of Diffusion Modelscited by 303
  25. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 258
  26. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  27. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
  28. 2021K-Net: Towards Unified Image Segmentationcited by 212
  29. 2017Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detectioncited by 209
  30. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  31. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
  32. 2019DoveNet: Deep Image Harmonization via Domain Verificationcited by 135
  33. 2018ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositingcited by 90
  34. 2019Improving the Harmony of the Composite Image by Spatial-Separated Attention Modulecited by 73
  35. 2017GP-GAN: Towards Realistic High-Resolution Image Blendingcited by 64
  36. 2021Making Images Real Again: A Comprehensive Survey on Deep Image Compositioncited by 63
  37. 2019Deep Image Blendingcited by 62
  38. 2021SSH: A Self-Supervised Framework for Image Harmonizationcited by 60

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 58; 2025: 76; 2026: 38.

182 papers.

Sort:

By area:

  1. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 307
  2. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  3. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 124
  4. 2023Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusioncited by 114
  5. 2024Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advancescited by 110
  6. 2023Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restorationcited by 88
  7. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmentercited by 76
  8. 2024IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representationcited by 64
  9. 2025Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Conceptscited by 62
  10. 2023Relightful Harmonization: Lighting-aware Portrait Background Replacementcited by 57
  11. 2023LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Promptscited by 54
  12. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 51
38 more
  1. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  2. 2025DBLoss: Decomposition-based Loss Function for Time Series Forecastingcited by 45
  3. 2025GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modelingcited by 44
  4. 2024STEREO: A Two-Stage Framework for Adversarially Robust Concept Erasing from Text-to-Image Diffusion Modelscited by 42
  5. 2025OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrievalcited by 40
  6. 2025Enhancing Time Series Forecasting through Selective Representation Spaces: A Patch Perspectivecited by 39
  7. 2024MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generationcited by 38
  8. 20243DIS: Depth-Driven Decoupled Instance Synthesis for Text-to-Image Generationcited by 37
  9. 2025ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMcited by 37
  10. 2024MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesiscited by 35
  11. 2025Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptationcited by 33
  12. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 32
  13. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  14. 2025ObjectMover: Generative Object Movement with Video Priorcited by 30
  15. 2025Towards Universal Modal Tracking with Online Dense Temporal Token Learningcited by 30
  16. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  17. 2025Aurora: Towards Universal Generative Multimodal Time Series Forecastingcited by 29
  18. 2025DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editingcited by 29
  19. 2025DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale Text-to-Image Modelscited by 28
  20. 2023Stable Diffusion Exposed: Gender Bias from Prompt to Imagecited by 25
  21. 2024Learning A Physical-aware Diffusion Model Based on Transformer for Underwater Image Enhancementcited by 25
  22. 2024Tuning-Free Image Customization with Image and Text Guidancecited by 25
  23. 2024Kandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Frameworkcited by 25
  24. 2024CPR: Retrieval Augmented Generation for Copyright Protectioncited by 24
  25. 2024Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shiftcited by 24
  26. 2026HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrievalcited by 24
  27. 2024Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perceptioncited by 23
  28. 2025From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspectivecited by 23
  29. 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 23
  30. 2025Exploring Decoupled Spatio-Temporal Consistency Learning and Self-Prompting Evolution for Self-Supervised Trackingcited by 23
  31. 2024HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awarenesscited by 21
  32. 2024KinMo: Kinematic-aware Human Motion Understanding and Generationcited by 21
  33. 2024PrimeComposer: Faster Progressively Combined Diffusion for Image Composition with Attention Steeringcited by 20
  34. 2024Thinking Outside the BBox: Unconstrained Generative Object Compositingcited by 19
  35. 2025UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Datasetcited by 18
  36. 2024Magic Insert: Style-Aware Drag-and-Dropcited by 17
  37. 2026ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrievalcited by 17
  38. 2025Concept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gatecited by 16

50 most cited of 182 All 182 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Cross-domain Compositing with Pretrained Diffusion Models
  2. 2023Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Models
  3. 2023Textual and Visual Prompt Fusion for Image Editing via Step-Wise Alignment
  4. 2026Training-free image inversion for one-step diffusion models
  5. 2023Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models
  6. 2024Diffusion Soup: Model Merging for Text-to-Image Diffusion Models
  7. 2024FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior
  8. 2022Versatile Diffusion: Text, Images and Variations All in One Diffusion Model