mlsanityData to 7 Oct 2026

MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing

arXiv 2306.10012 ↗First posted 16 Jun 2023cs.CV cs.AI cs.CLcited by 615

Not on the map. Why?

Cited by 150 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Text-guided image editing is widely needed in daily life, ranging from personal use to professional applications such as Photoshop. However, existing methods are either zero-shot or trained on an automatically synthesized dataset, which contains a high volume of noise. Thus, they still require lots of manual tuning to produce desirable outcomes in practice. To address this issue, we introduce MagicBrush (https://osu-nlp-group.github.io/MagicBrush/), the first large-scale, manually annotated dataset for instruction-guided real image editing that covers diverse scenarios: single-turn, multi-turn, mask-provided, and mask-free editing. MagicBrush comprises over 10K manually annotated triplets (source image, instruction, target image), which supports trainining large-scale text-guided image editing models. We fine-tune InstructPix2Pix on MagicBrush and show that the new model can produce much better images according to human evaluation. We further conduct extensive experiments to evaluate current image editing baselines from multiple dimensions including quantitative, qualitative, and human evaluations. The results reveal the challenging nature of our dataset and the gap between current baselines and real-world editing needs.

Key facts. Cited by 615 papers in the map (data through 7 Oct 2026). Builds on 38 of its 44 references that are in the map. First posted on arXiv 16 Jun 2023.

Builds on

38 of 44 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
26 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  3. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  5. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  6. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  7. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  8. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  9. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  10. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  11. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  12. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 229
  13. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 210
  14. 2016Learning Deep Representations of Fine-grained Visual Descriptionscited by 182
  15. 2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 172
  16. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  17. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  18. 2021Paint by Wordcited by 107
  19. 2017Semantic Image Synthesis via Adversarial Learningcited by 87
  20. 2018Tell, Draw, and Repeat: Generating and Modifying Images Based on Continual Linguistic Instructioncited by 61
  21. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  22. 2020Lightweight Generative Adversarial Networks for Text-Guided Image Manipulationcited by 33
  23. 2020SSCR: Iterative Language-Based Image Editing via Self-Supervised Counterfactual Reasoningcited by 32
  24. 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
  25. 2020Text as Neural Operator: Image Manipulation by Text Instructioncited by 26
  26. 2020Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructionscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 33; 2024: 126; 2025: 243; 2026: 213.

615 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2025Qwen-Image Technical Reportcited by 973
  3. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  4. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  5. 2025VACE: All-in-One Video Creation and Editingcited by 467
  6. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  7. 2024OmniGen: Unified Image Generationcited by 388
  8. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  9. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  10. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  11. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  12. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
38 more
  1. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  2. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  3. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  4. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  5. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
  6. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 196
  7. 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 191
  8. 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 184
  9. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 182
  10. 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Modelscited by 174
  11. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 165
  12. 2023InstructDiffusion: A Generalist Modeling Interface for Vision Taskscited by 162
  13. 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 151
  14. 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 148
  15. 2024OmniEdit: Building Image Editing Generalist Models Through Specialist Supervisioncited by 145
  16. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 132
  17. 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 131
  18. 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
  19. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  20. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
  21. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 111
  22. 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 107
  23. 2024Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advancescited by 105
  24. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editingcited by 103
  25. 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 100
  26. 2024SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editingcited by 97
  27. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  28. 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
  29. 2025ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generationcited by 94
  30. 2024DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editingcited by 92
  31. 2025LongCat-Image Technical Reportcited by 92
  32. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  33. 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 81
  34. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  35. 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedbackcited by 78
  36. 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Followingcited by 77
  37. 2023ImagenHub: Standardizing the evaluation of conditional image generation modelscited by 75
  38. 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 69

50 most cited of 615 All 615 (JSON)