# Custom-Edit: Text-Guided Image Editing with Customized Diffusion Models

arXiv: https://arxiv.org/abs/2305.15779 · HTML: https://mlsanity.com/paper/2305.15779/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 25 of its 28 references that are in the map.
- First posted on arXiv 25 May 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [Text2LIVE: Text-Driven Layered Image and Video Editing](https://mlsanity.com/paper/2204.02491/) (2022)
- [Few-shot Image Generation via Cross-domain Correspondence](https://mlsanity.com/paper/2104.06820/) (2021)

## Built on by (most cited first, 30 of 54)

- [Diffusion Model-Based Image Editing: A Survey](https://mlsanity.com/paper/2402.17525/) (2024)
- [VideoBooth: Diffusion-based Video Generation with Image Prompts](https://mlsanity.com/paper/2312.00777/) (2023)
- [Controllable Generation with Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2403.04279/) (2024)
- [DITTO: Diffusion Inference-Time T-Optimization for Music Generation](https://mlsanity.com/paper/2401.12179/) (2024)
- [VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondence](https://mlsanity.com/paper/2312.02087/) (2023)
- [ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuning](https://mlsanity.com/paper/2501.04698/) (2025)
- [Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control](https://mlsanity.com/paper/2405.12970/) (2024)
- [OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models](https://mlsanity.com/paper/2403.10983/) (2024)
- [MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning](https://mlsanity.com/paper/2311.13127/) (2023)
- [ControlCom: Controllable Image Composition using Diffusion Model](https://mlsanity.com/paper/2308.10040/) (2023)
- [Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices](https://mlsanity.com/paper/2410.11795/) (2024)
- [DreamEdit: Subject-driven Image Editing](https://mlsanity.com/paper/2306.12624/) (2023)
- [InstantSwap: Fast Customized Concept Swapping across Sharp Shape Differences](https://mlsanity.com/paper/2412.01197/) (2024)
- [SubjectDrive: Scaling Generative Data in Autonomous Driving via Subject Control](https://mlsanity.com/paper/2403.19438/) (2024)
- [ControlNet-XS: Rethinking the Control of Text-to-Image Diffusion Models as Feedback-Control Systems](https://mlsanity.com/paper/2312.06573/) (2023)
- [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024)
- [BELM: Bidirectional Explicit Linear Multi-step Sampler for Exact Inversion in Diffusion Models](https://mlsanity.com/paper/2410.07273/) (2024)
- [Watermark-embedded Adversarial Examples for Copyright Protection against Diffusion Models](https://mlsanity.com/paper/2404.09401/) (2024)
- [Source Prompt Disentangled Inversion for Boosting Image Editability with Diffusion Models](https://mlsanity.com/paper/2403.11105/) (2024)
- [IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models](https://mlsanity.com/paper/2311.06792/) (2023)
- [Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing](https://mlsanity.com/paper/2411.16832/) (2024)
- [DreamCom: Finetuning Text-guided Inpainting Model for Image Composition](https://mlsanity.com/paper/2309.15508/) (2023)
- [MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation](https://mlsanity.com/paper/2408.14899/) (2024)
- [Distillation of Diffusion Features for Semantic Correspondence](https://mlsanity.com/paper/2412.03512/) (2024)
- [SwapAnything: Enabling Arbitrary Object Swapping in Personalized Visual Editing](https://mlsanity.com/paper/2404.05717/) (2024)
- [Latent Drifting in Diffusion Models for Counterfactual Medical Image Synthesis](https://mlsanity.com/paper/2412.20651/) (2024)
- [On the Limitations of Vision-Language Models in Understanding Image Transforms](https://mlsanity.com/paper/2503.09837/) (2025)
- [PrimeDepth: Efficient Monocular Depth Estimation with a Stable Diffusion Preimage](https://mlsanity.com/paper/2409.09144/) (2024)
- [Cross Initialization for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2312.15905/) (2023)
- [MedEBench: Diagnosing Reliability in Text-Guided Medical Image Editing](https://mlsanity.com/paper/2506.01921/) (2025)

## Abstract (arXiv)

Text-to-image diffusion models can generate diverse, high-fidelity images based on user-provided text prompts. Recent research has extended these models to support text-guided image editing. While text guidance is an intuitive editing interface for users, it often fails to ensure the precise concept conveyed by users. To address this issue, we propose Custom-Edit, in which we (i) customize a diffusion model with a few reference images and then (ii) perform text-guided editing. Our key discovery is that customizing only language-relevant parameters with augmented prompts improves reference similarity significantly while maintaining source similarity. Moreover, we provide our recipe for each customization and editing process. We compare popular customization methods and validate our findings on two editing methods using various datasets.