mlsanityData to 14 Oct 2026

Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting

arXiv 2212.06909 ↗First posted 13 Dec 2022cs.CV cs.AIcited by 233

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Text-guided image editing can have a transformative impact in supporting creative applications. A key challenge is to generate edits that are faithful to input text prompts, while consistent with input images. We present Imagen Editor, a cascaded diffusion model built, by fine-tuning Imagen on text-guided image inpainting. Imagen Editor's edits are faithful to the text prompts, which is accomplished by using object detectors to propose inpainting masks during training. In addition, Imagen Editor captures fine details in the input image by conditioning the cascaded pipeline on the original high resolution image. To improve qualitative and quantitative evaluation, we introduce EditBench, a systematic benchmark for text-guided image inpainting. EditBench evaluates inpainting edits on natural and generated images exploring objects, attributes, and scenes. Through extensive human evaluation on EditBench, we find that object-masking during training leads to across-the-board improvements in text-image alignment -- such that Imagen Editor is preferred over DALL-E 2 and Stable Diffusion -- and, as a cohort, these models are better at object-rendering than text-rendering, and handle material/color/size attributes better than count/shape attributes.

Key facts. Cited by 233 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 39 of its 54 references that are in the map. First posted on arXiv 13 Dec 2022.

Builds on

39 of 54 references.

Closest ancestors · 8

Same direction, newest first

  1. 2022DiffEdit: Diffusion-based semantic image editing with mask guidance20 Oct
  2. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  7. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov
  8. 2021Palette: Image-to-Image Diffusion Models10 Nov

Other references · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2022Classifier-Free Diffusion Guidancecited by 5,847
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  9. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  10. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  11. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  12. 2016Sequence-Level Knowledge Distillationcited by 801
19 more
  1. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  2. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  3. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 602
  4. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  5. 2018Generative Image Inpainting with Contextual Attentioncited by 580
  6. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  7. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  8. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  9. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  10. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 307
  11. 2020On the Binding Problem in Artificial Neural Networkscited by 233
  12. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  13. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 135
  14. 2021Paint by Wordcited by 107
  15. 2021ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generationcited by 41
  16. 2022ManiTrans: Entity-Level Text-Guided Image Manipulation via Token-wise Semantic Alignment and Generationcited by 13
  17. 2022LECA: A Learned Approach for Efficient Cover-agnostic Watermarkingcited by 13
  18. 2022DE-Net: Dynamic Text-guided Image Editing Adversarial Networkscited by 12
  19. 2020Rank-smoothed Pairwise Learning In Perceptual Quality Assessmentcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 42; 2024: 73; 2025: 78; 2026: 40.

233 papers.

Sort:

By area:

  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  2. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  3. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  4. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  5. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 327
  6. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
  7. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 304
  8. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  9. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
  10. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  11. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 231
  12. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
38 more
  1. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  2. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 202
  3. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 185
  4. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 177
  5. 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 169
  6. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 161
  7. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Modelscited by 157
  8. 2023Rich Human Feedback for Text-to-Image Generationcited by 141
  9. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 135
  10. 2024DOCCI: Descriptions of Connected and Contrasting Imagescited by 132
  11. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  12. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 113
  13. 2024Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advancescited by 110
  14. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 108
  15. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 108
  16. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 108
  17. 2023Monocular Depth Estimation using Diffusion Modelscited by 87
  18. 2024Video as the New Language for Real-World Decision Makingcited by 87
  19. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  20. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  21. 2023AVID: Any-Length Video Inpainting with Diffusion Modelcited by 82
  22. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matchingcited by 76
  23. 2024ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertioncited by 71
  24. 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 71
  25. 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
  26. 2023HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Modelscited by 67
  27. 2024I2EBench: A Comprehensive Benchmark for Instruction-based Image Editingcited by 63
  28. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 61
  29. 2023Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Modelcited by 60
  30. 2023RealFill: Reference-Driven Generation for Authentic Image Completioncited by 59
  31. 2023CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layoutcited by 57
  32. 2023GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generationcited by 56
  33. 2023PanoDiffusion: 360-degree Panorama Outpainting via Diffusioncited by 51
  34. 2024CoCoCo: Improving Text-Guided Video Inpainting for Better Consistency, Controllability and Compatibilitycited by 50
  35. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 45
  36. 2025MiniMax-Remover: Taming Bad Noise Helps Video Object Removalcited by 45
  37. 2024Deep Learning-based Image and Video Inpainting: A Surveycited by 44
  38. 2023Prompt-In-Prompt Learning for Universal Image Restorationcited by 43

50 most cited of 233 All 233 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Text-guided Object Inpainting with Semantic Pre-inpainting
  2. 2023EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methods
  3. 2025PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models
  4. 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs
  5. 2024Image Inpainting Models are Effective Tools for Instruction-guided Image Editing
  6. 2023DreamInpainter: Text-Guided Subject-Driven Image Inpainting with Diffusion Models
  7. 2023Magicremover: Tuning-free Text-guided Image inpainting with Diffusion Models
  8. 2024Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing