# Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions

arXiv: https://arxiv.org/abs/2008.01576 · HTML: https://mlsanity.com/paper/2008.01576/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2020.

## Built on by (most cited first, 25 of 25)

- [MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing](https://mlsanity.com/paper/2306.10012/) (2023)
- [VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance](https://mlsanity.com/paper/2204.08583/) (2022)
- [OmniEdit: Building Image Editing Generalist Models Through Specialist Supervision](https://mlsanity.com/paper/2411.07199/) (2024)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing](https://mlsanity.com/paper/2504.02826/) (2025)
- [DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generation](https://mlsanity.com/paper/2402.11929/) (2024)
- [Alchemist: Parametric Control of Material Properties with Diffusion Models](https://mlsanity.com/paper/2312.02970/) (2023)
- [Language-Driven Image Style Transfer](https://mlsanity.com/paper/2106.00178/) (2021)
- [LIME: Localized Image Editing via Attention Regularization in Diffusion Models](https://mlsanity.com/paper/2312.09256/) (2023)
- [Region-Aware Diffusion for Zero-shot Text-driven Image Editing](https://mlsanity.com/paper/2302.11797/) (2023)
- [Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learning](https://mlsanity.com/paper/2309.01246/) (2023)
- [DE-Net: Dynamic Text-guided Image Editing Adversarial Networks](https://mlsanity.com/paper/2206.01160/) (2022)
- [MVP-SEG: Multi-View Prompt Learning for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2304.06957/) (2023)
- [InstructVEdit: A Holistic Approach for Instructional Video Editing](https://mlsanity.com/paper/2503.17641/) (2025)
- [Language-based Photo Color Adjustment for Graphic Designs](https://mlsanity.com/paper/2308.03059/) (2023)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)
- [ScaleEdit-12M: Scaling Open-Source Image Editing Data Generation via Multi-Agent Framework](https://mlsanity.com/paper/2603.20644/) (2026)
- [SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing](https://mlsanity.com/paper/2112.00180/) (2021)
- [Content-aware Tile Generation using Exterior Boundary Inpainting](https://mlsanity.com/paper/2409.14184/) (2024)
- [HorizonWeaver: Generalizable Multi-Level Semantic Editing for Driving Scenes](https://mlsanity.com/paper/2604.04887/) (2026)
- [Each Attribute Matters: Contrastive Attention for Sentence-based Image Editing](https://mlsanity.com/paper/2110.11159/) (2021)
- [Target-Free Text-guided Image Manipulation](https://mlsanity.com/paper/2211.14544/) (2022)
- [UIP2P: Unsupervised Instruction-based Image Editing via Edit Reversibility Constraint](https://mlsanity.com/paper/2412.15216/) (2024)
- [PixLens: A Novel Framework for Disentangled Evaluation in Diffusion-Based Image Editing with Object Detection + SAM](https://mlsanity.com/paper/2410.05710/) (2024)
- [Understanding the Implicit User Intention via Reasoning with Large Language Model for Image Editing](https://mlsanity.com/paper/2510.27335/) (2025)

## Abstract (arXiv)

We propose a novel algorithm, named Open-Edit, which is the first attempt on open-domain image manipulation with open-vocabulary instructions. It is a challenging task considering the large variation of image domains and the lack of training supervision. Our approach takes advantage of the unified visual-semantic embedding space pretrained on a general image-caption dataset, and manipulates the embedded visual features by applying text-guided vector arithmetic on the image feature maps. A structure-preserving image decoder then generates the manipulated images from the manipulated feature maps. We further propose an on-the-fly sample-specific optimization approach with cycle-consistency constraints to regularize the manipulated images and force them to preserve details of the source images. Our approach shows promising results in manipulating open-vocabulary color, texture, and high-level attributes for various scenarios of open-domain images.