# Tuning-Free Image Customization with Image and Text Guidance

arXiv: https://arxiv.org/abs/2403.12658 · HTML: https://mlsanity.com/paper/2403.12658/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 42 of its 47 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 42)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation](https://mlsanity.com/paper/2302.13848/) (2023)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [SVDiff: Compact Parameter Space for Diffusion Fine-Tuning](https://mlsanity.com/paper/2303.11305/) (2023)
- [SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model](https://mlsanity.com/paper/2212.05034/) (2022)

## Built on by (most cited first, 25 of 25)

- [FreeEdit: Mask-free Reference-based Image Editing with Multi-modal Instruction](https://mlsanity.com/paper/2409.18071/) (2024)
- [InstantSwap: Fast Customized Concept Swapping across Sharp Shape Differences](https://mlsanity.com/paper/2412.01197/) (2024)
- [Dual-Interrelated Diffusion Model for Few-Shot Anomaly Image Generation](https://mlsanity.com/paper/2408.13509/) (2024)
- [Does FLUX Already Know How to Perform Physically Plausible Image Composition?](https://mlsanity.com/paper/2509.21278/) (2025)
- [A Survey on Personalized Content Synthesis with Diffusion Models](https://mlsanity.com/paper/2405.05538/) (2024)
- [UniCombine: Unified Multi-Conditional Combination with Diffusion Transformer](https://mlsanity.com/paper/2503.09277/) (2025)
- [Pruning then Reweighting: Towards Data-Efficient Training of Diffusion Models](https://mlsanity.com/paper/2409.19128/) (2024)
- [MagicTailor: Component-Controllable Personalization in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2410.13370/) (2024)
- [MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing](https://mlsanity.com/paper/2502.21291/) (2025)
- [ART3D: 3D Gaussian Splatting for Text-Guided Artistic Scenes Generation](https://mlsanity.com/paper/2405.10508/) (2024)
- [OmniVTON: Training-Free Universal Virtual Try-On](https://mlsanity.com/paper/2507.15037/) (2025)
- [MVOC: a training-free multiple video object composition method with diffusion models](https://mlsanity.com/paper/2406.15829/) (2024)
- [In-Context Brush: Zero-shot Customized Subject Insertion with Context-Aware Latent Space Manipulation](https://mlsanity.com/paper/2505.20271/) (2025)
- [Multi-party Collaborative Attention Control for Image Customization](https://mlsanity.com/paper/2505.01428/) (2025)
- [LDGen: Enhancing Text-to-Image Synthesis via Large Language Model-Driven Language Representation](https://mlsanity.com/paper/2502.18302/) (2025)
- [DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting](https://mlsanity.com/paper/2411.17223/) (2024)
- [Challenges and Opportunities in 3D Content Generation](https://mlsanity.com/paper/2405.15335/) (2024)
- [Zero-Shot Personalized Camera Motion Control for Image-to-Video Synthesis](https://mlsanity.com/paper/2504.09472/) (2025)
- [Crafting Your Evolving Dreams: Concept-Incremental Versatile Customization](https://mlsanity.com/paper/2606.04797/) (2026)
- [Single Image Iterative Subject-driven Generation and Editing](https://mlsanity.com/paper/2503.16025/) (2025)
- [EraseLoRA: MLLM-Driven Foreground Exclusion and Background Subtype Aggregation for Dataset-Free Object Removal](https://mlsanity.com/paper/2512.21545/) (2025)
- [OmniVTON++: Training-Free Universal Virtual Try-On with Principal Pose Guidance](https://mlsanity.com/paper/2602.14552/) (2026)
- [Training-Free Image Editing with Visual Context Integration and Concept Alignment](https://mlsanity.com/paper/2604.04487/) (2026)
- [Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generation](https://mlsanity.com/paper/2605.26111/) (2026)
- [Rethinking Camouflage Image Generation towards a Training-Free Paradigm](https://mlsanity.com/paper/2609.14377/) (2026)

## Abstract (arXiv)

Despite significant advancements in image customization with diffusion models, current methods still have several limitations: 1) unintended changes in non-target areas when regenerating the entire image; 2) guidance solely by a reference image or text descriptions; and 3) time-consuming fine-tuning, which limits their practical application. In response, we introduce a tuning-free framework for simultaneous text-image-guided image customization, enabling precise editing of specific image regions within seconds. Our approach preserves the semantic features of the reference image subject while allowing modification of detailed attributes based on text descriptions. To achieve this, we propose an innovative attention blending strategy that blends self-attention features in the UNet decoder during the denoising process. To our knowledge, this is the first tuning-free method that concurrently utilizes text and image guidance for image customization in specific regions. Our approach outperforms previous methods in both human and quantitative evaluations, providing an efficient solution for various practical applications, such as image synthesis, design, and creative photography.