# DeltaEdit: Exploring Text-free Training for Text-Driven Image Manipulation

arXiv: https://arxiv.org/abs/2303.06285 · HTML: https://mlsanity.com/paper/2303.06285/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 48 references that are in the map.
- First posted on arXiv 11 Mar 2023.

## Builds on (most cited first, 30 of 38)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1710.10916/) (2017)
- [ManiGAN: Text-Guided Image Manipulation](https://mlsanity.com/paper/1912.06203/) (2019)
- [Domain Adaptation via Prompt Learning](https://mlsanity.com/paper/2202.06687/) (2022)
- [VinVL: Revisiting Visual Representations in Vision-Language Models](https://mlsanity.com/paper/2101.00529/) (2021)
- [Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language](https://mlsanity.com/paper/1810.11919/) (2018)
- [Semantic Image Synthesis via Adversarial Learning](https://mlsanity.com/paper/1707.06873/) (2017)
- [HairCLIP: Design Your Hair by Text and Reference Image](https://mlsanity.com/paper/2112.05142/) (2021)
- [FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimization](https://mlsanity.com/paper/2112.01573/) (2021)
- [Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks](https://mlsanity.com/paper/2110.08398/) (2021)
- [StyleMC: Multi-Channel Based Fast Text-Guided Image Generation and Manipulation](https://mlsanity.com/paper/2112.08493/) (2021)

## Built on by (most cited first, 27 of 27)

- [Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulation](https://mlsanity.com/paper/2312.10113/) (2023)
- [FreeDrag: Feature Dragging for Reliable Point-based Image Editing](https://mlsanity.com/paper/2307.04684/) (2023)
- [DiffusionAgent: Navigating Expert Models for Agentic Image Generation](https://mlsanity.com/paper/2401.10061/) (2024)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [InstructP2P: Learning to Edit 3D Point Clouds with Text Instructions](https://mlsanity.com/paper/2306.07154/) (2023)
- [Text-guided Eyeglasses Manipulation with Spatial Constraints](https://mlsanity.com/paper/2304.12539/) (2023)
- [AdaptiveDrag: Semantic-Driven Dragging on Diffusion-Based Image Editing](https://mlsanity.com/paper/2410.12696/) (2024)
- [RunawayEvil: Jailbreaking the Image-to-Video Generative Models](https://mlsanity.com/paper/2512.06674/) (2025)
- [MagicFace: High-Fidelity Facial Expression Editing with Action-Unit Control](https://mlsanity.com/paper/2501.02260/) (2025)
- [Unlocking Attributes' Contribution to Successful Camouflage: A Combined Textual and VisualAnalysis Strategy](https://mlsanity.com/paper/2408.12086/) (2024)
- [Training-free Boost for Open-Vocabulary Object Detection with Confidence Aggregation](https://mlsanity.com/paper/2404.08603/) (2024)
- [GCA-3D: Towards Generalized and Consistent Domain Adaptation of 3D Generators](https://mlsanity.com/paper/2412.15491/) (2024)
- [DeltaSpace: A Semantic-aligned Feature Space for Flexible Text-guided Image Editing](https://mlsanity.com/paper/2310.08785/) (2023)
- [MasterWeaver: Taming Editability and Face Identity for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2405.05806/) (2024)
- [The Devil is in Attention Sharing: Improving Complex Non-rigid Image Editing Faithfulness via Attention Synergy](https://mlsanity.com/paper/2512.14423/) (2025)
- [GaFET: Learning Geometry-aware Facial Expression Translation from In-The-Wild Images](https://mlsanity.com/paper/2308.03413/) (2023)
- [RS-Corrector: Correcting the Racial Stereotypes in Latent Diffusion Models](https://mlsanity.com/paper/2312.04810/) (2023)
- [HyperEditor: Achieving Both Authenticity and Cross-Domain Capability in Image Editing via Hypernetworks](https://mlsanity.com/paper/2312.13537/) (2023)
- [AUEditNet: Dual-Branch Facial Action Unit Intensity Manipulation with Implicit Disentanglement](https://mlsanity.com/paper/2404.05063/) (2024)
- [Cyclic Vision-Language Manipulator: Towards Reliable and Fine-Grained Image Interpretation for Automated Report Generation](https://mlsanity.com/paper/2411.05261/) (2024)
- [HyperGAN-CLIP: A Unified Framework for Domain Adaptation, Image Synthesis and Manipulation](https://mlsanity.com/paper/2411.12832/) (2024)
- [S2Edit: Text-Guided Image Editing with Precise Semantic and Spatial Control](https://mlsanity.com/paper/2507.04584/) (2025)
- [UniHDA: A Unified and Versatile Framework for Multi-Modal Hybrid Domain Adaptation](https://mlsanity.com/paper/2401.12596/) (2024)
- [Dark Miner: Defend against undesirable generation for text-to-image diffusion models](https://mlsanity.com/paper/2409.17682/) (2024)
- [High-Fidelity 3D Facial Avatar Synthesis with Controllable Fine-Grained Expressions](https://mlsanity.com/paper/2603.14781/) (2026)
- [ExDet: Open-Domain Open-Vocabulary Detection with Cross-modal Extrapolation and Rectification](https://mlsanity.com/paper/2606.09360/) (2026)
- [Debias Anything: Fairness with Diversity without Supervision in Diffusion Models](https://mlsanity.com/paper/2610.01815/) (2026)

## Abstract (arXiv)

Text-driven image manipulation remains challenging in training or inference flexibility. Conditional generative models depend heavily on expensive annotated training data. Meanwhile, recent frameworks, which leverage pre-trained vision-language models, are limited by either per text-prompt optimization or inference-time hyper-parameters tuning. In this work, we propose a novel framework named textitDeltaEdit to address these problems. Our key idea is to investigate and identify a space, namely delta image and text space that has well-aligned distribution between CLIP visual feature differences of two images and CLIP textual embedding differences of source and target texts. Based on the CLIP delta space, the DeltaEdit network is designed to map the CLIP visual features differences to the editing directions of StyleGAN at training phase. Then, in inference phase, DeltaEdit predicts the StyleGAN's editing directions from the differences of the CLIP textual features. In this way, DeltaEdit is trained in a text-free manner. Once trained, it can well generalize to various text prompts for zero-shot inference without bells and whistles.