# GIE-Bench: Towards Grounded Evaluation for Text-Guided Image Editing arXiv: https://arxiv.org/abs/2505.11493 ยท HTML: https://mlsanity.com/paper/2505.11493/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Builds on 14 of its 25 references that are in the map. - First posted on arXiv 16 May 2025. ## Builds on (most cited first, 14 of 14) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023) - [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022) - [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022) - [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023) - [MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing](https://mlsanity.com/paper/2306.10012/) (2023) - [Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting](https://mlsanity.com/paper/2212.06909/) (2022) - [Guiding Instruction-based Image Editing via Multimodal Large Language Models](https://mlsanity.com/paper/2309.17102/) (2023) - [Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2310.18235/) (2023) - [I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing](https://mlsanity.com/paper/2408.14180/) (2024) - [Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulation](https://mlsanity.com/paper/2312.10113/) (2023) - [EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluation](https://mlsanity.com/paper/2412.18150/) (2024) - [InstructAny2Pix: Flexible Visual Editing via Multimodal Instruction Following](https://mlsanity.com/paper/2312.06738/) (2023) - [IE-Bench: Advancing the Measurement of Text-Driven Image Editing for Human Perception Alignment](https://mlsanity.com/paper/2501.09927/) (2025) ## Built on by (most cited first, 19 of 19) - [Slot-MLLM: Object-Centric Visual Tokenization for Multimodal LLM](https://mlsanity.com/paper/2505.17726/) (2025) - [I2I-Bench: A Comprehensive Benchmark Suite for Image-to-Image Editing Models](https://mlsanity.com/paper/2512.04660/) (2025) - [From Image Generation to Infrastructure Design: a Multi-agent Pipeline for Street Design Generation](https://mlsanity.com/paper/2509.05469/) (2025) - [CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructions](https://mlsanity.com/paper/2603.26174/) (2026) - [FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation](https://mlsanity.com/paper/2604.13491/) (2026) - [VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents](https://mlsanity.com/paper/2602.00122/) (2026) - [GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editing](https://mlsanity.com/paper/2603.21176/) (2026) - [MULTITEXTEDIT: Benchmarking Cross-Lingual Degradation in Text-in-Image Editing](https://mlsanity.com/paper/2605.08163/) (2026) - [MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching](https://mlsanity.com/paper/2606.01985/) (2026) - [DuET: Dual Expert Trajectories for Diffusion Image Editing](https://mlsanity.com/paper/2606.13303/) (2026) - [DLEBench: Evaluating Small-scale Object Editing Ability for Instruction-based Image Editing Model](https://mlsanity.com/paper/2602.23622/) (2026) - [Editor's Choice: Evaluating Abstract Intent in Image Editing through Atomic Entity Analysis](https://mlsanity.com/paper/2605.14842/) (2026) - [Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews](https://mlsanity.com/paper/2608.02841/) (2026) - [A Large-scale Evaluation of Text-guided Models for Facial Editing](https://mlsanity.com/paper/2608.28802/) (2026) - [SolveEdit: Benchmarking Visual Problem Solving in Generative Models](https://mlsanity.com/paper/2609.35504/) (2026) - [When Text-to-Image Helps Editing: The Effects of Conditioning During Denoising](https://mlsanity.com/paper/2610.01681/) (2026) - [Beyond Plausibility: Verifiable Fine-Grained Image Editing on Structured Assets](https://mlsanity.com/paper/2610.04381/) (2026) - [Scalable Minimal-Change Learning for Controllable Image Editing](https://mlsanity.com/paper/2610.06021/) (2026) - [Diagnose-to-Generate: Eliminating Structural Distortions in Flow Models via Patch-Level Grounded Alignment](https://mlsanity.com/paper/2610.17882/) (2026) ## Abstract (arXiv) Editing images using natural language instructions has become a natural and expressive way to modify visual content; yet, evaluating the performance of such models remains challenging. Existing evaluation approaches often rely on image-text similarity metrics like CLIP, which lack precision. In this work, we introduce a new benchmark designed to evaluate text-guided image editing models in a more grounded manner, along two critical dimensions: (i) functional correctness, assessed via automatically generated multiple-choice questions that verify whether the intended change was successfully applied; and (ii) image content preservation, which ensures that non-targeted regions of the image remain visually consistent using an object-aware masking technique and preservation scoring. The benchmark includes over 1000 high-quality editing examples across 20 diverse content categories, each annotated with detailed editing instructions, evaluation questions, and spatial object masks. We conduct a large-scale study comparing GPT-Image-1, the latest flagship in the text-guided image editing space, against several state-of-the-art editing models, and validate our automatic metrics against human ratings. Results show that GPT-Image-1 leads in instruction-following accuracy, but often over-modifies irrelevant image regions, highlighting a key trade-off in the current model behavior. GIE-Bench provides a scalable, reproducible framework for advancing more accurate evaluation of text-guided image editing.