# UniTune: Text-Driven Image Editing by Fine Tuning a Diffusion Model on a Single Image

arXiv: https://arxiv.org/abs/2210.09477 · HTML: https://mlsanity.com/paper/2210.09477/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 37 of its 46 references that are in the map.
- First posted on arXiv 17 Oct 2022.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 37)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [Text2LIVE: Text-Driven Layered Image and Video Editing](https://mlsanity.com/paper/2204.02491/) (2022)

## Built on by (most cited first, 30 of 42)

- [Training-Free Layout Control with Cross-Attention Guidance](https://mlsanity.com/paper/2304.03373/) (2023)
- [Negative-prompt Inversion: Fast Image Inversion for Editing with Text-guided Diffusion Models](https://mlsanity.com/paper/2305.16807/) (2023)
- [RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Models](https://mlsanity.com/paper/2312.04524/) (2023)
- [ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models](https://mlsanity.com/paper/2305.16225/) (2023)
- [TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models](https://mlsanity.com/paper/2408.00735/) (2024)
- [I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing](https://mlsanity.com/paper/2408.14180/) (2024)
- [NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models](https://mlsanity.com/paper/2312.05390/) (2023)
- [Shape-Guided Diffusion with Inside-Outside Attention](https://mlsanity.com/paper/2212.00210/) (2022)
- [Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignment](https://mlsanity.com/paper/2403.12965/) (2024)
- [Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices](https://mlsanity.com/paper/2410.11795/) (2024)
- [PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions](https://mlsanity.com/paper/2308.05184/) (2023)
- [Attention Calibration for Disentangled Text-to-Image Personalization](https://mlsanity.com/paper/2403.18551/) (2024)
- [Token Caching for Diffusion Transformer Acceleration](https://mlsanity.com/paper/2409.18523/) (2024)
- [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024)
- [Noise Map Guidance: Inversion with Spatial Context for Real Image Editing](https://mlsanity.com/paper/2402.04625/) (2024)
- [CLIPDrag: Combining Text-based and Drag-based Instructions for Image Editing](https://mlsanity.com/paper/2410.03097/) (2024)
- [GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting](https://mlsanity.com/paper/2410.05259/) (2024)
- [Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Data](https://mlsanity.com/paper/2503.01167/) (2025)
- [PostEdit: Posterior Sampling for Efficient Zero-Shot Image Editing](https://mlsanity.com/paper/2410.04844/) (2024)
- [Cut-and-Paste: Subject-Driven Video Editing with Attention Control](https://mlsanity.com/paper/2311.11697/) (2023)
- [DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model](https://mlsanity.com/paper/2505.12427/) (2025)
- [Data Redaction from Conditional Generative Models](https://mlsanity.com/paper/2305.11351/) (2023)
- [PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models](https://mlsanity.com/paper/2502.04050/) (2025)
- [FreeInv: Free Lunch for Improving DDIM Inversion](https://mlsanity.com/paper/2503.23035/) (2025)
- [EditInfinity: Image Editing with Binary-Quantized Generative Models](https://mlsanity.com/paper/2510.20217/) (2025)
- [EditScribe: Non-Visual Image Editing with Natural Language Verification Loops](https://mlsanity.com/paper/2408.06632/) (2024)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)
- [EditCLIP: Representation Learning for Image Editing](https://mlsanity.com/paper/2503.20318/) (2025)
- [The Curious Case of End Token: A Zero-Shot Disentangled Image Editing using CLIP](https://mlsanity.com/paper/2406.00457/) (2024)
- [CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing](https://mlsanity.com/paper/2402.17624/) (2024)

## Abstract (arXiv)

Text-driven image generation methods have shown impressive results recently, allowing casual users to generate high quality images by providing textual descriptions. However, similar capabilities for editing existing images are still out of reach. Text-driven image editing methods usually need edit masks, struggle with edits that require significant visual changes and cannot easily keep specific details of the edited portion. In this paper we make the observation that image-generation models can be converted to image-editing models simply by fine-tuning them on a single image. We also show that initializing the stochastic sampler with a noised version of the base image before the sampling and interpolating relevant details from the base image after sampling further increase the quality of the edit operation. Combining these observations, we propose UniTune, a novel image editing method. UniTune gets as input an arbitrary image and a textual edit description, and carries out the edit while maintaining high fidelity to the input image. UniTune does not require additional inputs, like masks or sketches, and can perform multiple edits on the same image without retraining. We test our method using the Imagen model in a range of different use cases. We demonstrate that it is broadly applicable and can perform a surprisingly wide range of expressive editing operations, including those requiring significant visual changes that were previously impossible.