Beyond Plausibility: Verifiable Fine-Grained Image Editing on Structured Assets
Not on the map. Why?
Abstract (arXiv)
Fine-grained image editing requires more than producing a visually plausible result: an editor must execute the requested attribute change precisely while leaving everything else intact. However, existing benchmarks leave a critical gap between realism and verifiability: benchmarks built on realistic images typically rely on human or vision--language model judgments, while deterministic evaluation has largely focused on synthetic shape canvases, with application-oriented extensions primarily limited to charts. This makes it difficult to determine precisely how much of a requested edit was executed, where unintended changes occurred, and whether small differences between models reflect genuine editing capability or evaluator uncertainty. To bridge this gap, we present VeriEdit-Bench, a benchmark for fine-grained, instruction-faithful image editing across realistic structured assets with deterministic, four-axis evaluation. Its 1,740 cases are compiled from the source code of 153 Scalable Vector Graphics (SVG) graphics, charts, web interfaces, and presentation slides. Controlled source-code edits preserve the original visual context while yielding exact target images, pixel-level edit masks, and explicit edit specifications, enabling reproducible scoring along four axes: edit fidelity, preservation, localization, and magnitude. Evaluating eleven editors, we find that even the strongest model remains far from full credit; rankings for the same recoloring operation reverse between charts and SVG graphics; and outputs with similar pixel-accuracy profiles can still differ substantially in localization and change magnitude. This decomposition yields graded, verifiable feedback and exposes model-specific capability and failure profiles that holistic scores or evaluator-dependent judgments may obscure.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 45 references that are in the map. First posted on arXiv 3 Oct 2026.
Builds on
40 of 45 references.
References · 40
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2025Qwen-Image Technical Reportcited by 1,044
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
- 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
- 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
- 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 163
- 2026Kimi K3: Open Frontier Intelligencecited by 138
- 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 129
28 more
- 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 72
- 2025AutoPresent: Designing Structured Visuals from Scratchcited by 68
- 2024I2EBench: A Comprehensive Benchmark for Instruction-based Image Editingcited by 63
- 2023EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methodscited by 54
- 2024Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Modelscited by 38
- 2026FireRed-Image-Edit-1.0 Technical Reportcited by 36
- 2024From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editingcited by 31
- 2025textttComplex-Edit: CoT-Like Instruction Generation for Complexity-Controllable Image Editing Benchmarkcited by 27
- 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 25
- 2025LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMscited by 24
- 2025Factuality Matters: When Image Generation and Editing Meet Structured Visualscited by 20
- 2025GIE-Bench: Towards Grounded Evaluation for Text-Guided Image Editingcited by 19
- 2025SVGEditBench V2: A Benchmark for Instruction-based SVG Editingcited by 11
- 2025Talk to Your Slides: High-Efficiency Slide Editing via Language-Driven Structured Data Manipulationcited by 11
- 2025Charts Are Not Images: On the Challenges of Scientific Chart Editingcited by 11
- 2025OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilitiescited by 9
- 2026VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editingcited by 9
- 2026Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysiscited by 8
- 2026VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluationcited by 8
- 2025VectorEdits: A Dataset and Benchmark for Instruction-Based Editing of Vector Graphicscited by 5
- 2026I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editingcited by 5
- 2026PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Taskscited by 4
- 2026IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Frameworkcited by 4
- 2026ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editingcited by 4
- 2026VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documentscited by 2
- 2026PaintBench: Deterministic Evaluation of Precise Visual Editingcited by 2
- 2026DisciplineGen-1M: A Large-Scale Dataset for Multidisciplinary Visual Generation and Editingcited by 1
- 2026Vector-Bench: Can Models Surgically Edit SVG Code?cited by 1
Built on by
Not cited yet.