CompBench: Benchmarking Complex Instruction-guided Image Editing
Not on the map. Why?
Abstract (arXiv)
While real-world applications increasingly demand intricate scene manipulation, existing instruction-guided image editing benchmarks often oversimplify task complexity and lack comprehensive, fine-grained instructions. To bridge this gap, we introduce CompBench, a large-scale benchmark specifically designed for complex instruction-guided image editing. CompBench features challenging editing scenarios that incorporate fine-grained instruction following, spatial and contextual reasoning, thereby enabling comprehensive evaluation of image editing models' precise manipulation capabilities. To construct CompBench, we propose an MLLM-human collaborative framework with tailored task pipelines. Furthermore, we propose an instruction decoupling strategy that disentangles editing intents into four key dimensions: location, appearance, dynamics, and objects, ensuring closer alignment between instructions and complex editing requirements. Extensive evaluations reveal that CompBench exposes fundamental limitations of current image editing models and provides critical insights for the development of next-generation instruction-guided image editing systems. Our project page is available at https://comp-bench.github.io/.
Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 18 May 2025.
Builds on
46 of 59 references.
References · 46
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023Visual Instruction Tuningcited by 7,869
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
34 more
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2024GPT-4o System Cardcited by 4,588
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2021Zero-Shot Text-to-Image Generationcited by 4,288
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
- 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
- 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
- 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 522
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
- 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
- 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
- 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 193
- 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 185
- 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Modelscited by 177
- 2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 173
- 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 169
- 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editingcited by 104
- 2024SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editingcited by 97
- 2024I2EBench: A Comprehensive Benchmark for Instruction-based Image Editingcited by 63
- 2024Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsificationcited by 59
- 2024EditWorld: Simulating World Dynamics for Instruction-Following Image Editingcited by 38
- 2025textttComplex-Edit: CoT-Like Instruction Generation for Complexity-Controllable Image Editing Benchmarkcited by 27
- 2025TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generationcited by 13
- 2025LLaVA-RadZ: Can Multimodal Large Language Models Effectively Tackle Zero-shot Radiology Recognition?cited by 6
Built on by
Citing papers per year: 2025: 3; 2026: 14.
17 papers.
Sort:
By area:
- 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 25
- 2025MIRA: Multimodal Iterative Reasoning Agent for Image Editingcited by 17
- 2026UniT: Unified Multimodal Chain-of-Thought Test-time Scalingcited by 8
- 2025I2I-Bench: A Comprehensive Benchmark Suite for Image-to-Image Editing Modelscited by 7
- 2026Generative Visual Chain-of-Thought for Image Editingcited by 3
- 2026CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructionscited by 3
- 2026CAMEO: A Conditional and Quality-Aware Multi-Agent Image Editing Orchestratorcited by 3
- 2026VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documentscited by 2
- 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editingcited by 2
- 2026VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-Oncited by 1
- 2026Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Modelscited by 0
- 2026Editor's Choice: Evaluating Abstract Intent in Image Editing through Atomic Entity Analysiscited by 0
5 more
- 2026HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editingcited by 0
- 2026CV-Arena: An Open Benchmark for Instructional Computer Vision Problem Solving with Human-AI Collaborative Preferencescited by 0
- 2026Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applicationscited by 0
- 2026Scalable Minimal-Change Learning for Controllable Image Editingcited by 0
- 2026Pixels or Code? Benchmarking Instruction-Guided Visual Artifact Operationcited by 0
- 2026Pixels or Code? Benchmarking Instruction-Guided Visual Artifact Operation12 Oct 2026
- 2026Scalable Minimal-Change Learning for Controllable Image Editing5 Oct 2026
- 2026Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications28 Jul 2026
- 2026CV-Arena: An Open Benchmark for Instructional Computer Vision Problem Solving with Human-AI Collaborative Preferences30 May 2026
- 2026HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editing17 May 2026
- 2026Editor's Choice: Evaluating Abstract Intent in Image Editing through Atomic Entity Analysis14 May 2026
- 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editing10 Apr 2026
- 2026CAMEO: A Conditional and Quality-Aware Multi-Agent Image Editing Orchestrator3 Apr 2026
- 2026CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructions27 Mar 2026
- 2026Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Models16 Mar 2026
- 2026VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-On12 Mar 2026
- 2026Generative Visual Chain-of-Thought for Image Editing2 Mar 2026
5 more
- 2026UniT: Unified Multimodal Chain-of-Thought Test-time Scaling12 Feb 2026
- 2026VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents27 Jan 2026
- 2025I2I-Bench: A Comprehensive Benchmark Suite for Image-to-Image Editing Models4 Dec 2025
- 2025MIRA: Multimodal Iterative Reasoning Agent for Image Editing26 Nov 2025
- 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmark3 Nov 2025
- 2026UniT: Unified Multimodal Chain-of-Thought Test-time Scalingcited by 8
- 2026Generative Visual Chain-of-Thought for Image Editingcited by 3
- 2026Generative Visual Chain-of-Thought for Image Editing2 Mar 2026
- 2026UniT: Unified Multimodal Chain-of-Thought Test-time Scaling12 Feb 2026
- 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 25
- 2025MIRA: Multimodal Iterative Reasoning Agent for Image Editingcited by 17
- 2025I2I-Bench: A Comprehensive Benchmark Suite for Image-to-Image Editing Modelscited by 7
- 2026CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructionscited by 3
- 2026CAMEO: A Conditional and Quality-Aware Multi-Agent Image Editing Orchestratorcited by 3
- 2026VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documentscited by 2
- 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editingcited by 2
- 2026VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-Oncited by 1
- 2026Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Modelscited by 0
- 2026Editor's Choice: Evaluating Abstract Intent in Image Editing through Atomic Entity Analysiscited by 0
- 2026HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editingcited by 0
- 2026Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applicationscited by 0
12 most cited of 14 All 14 (JSON)
- 2026Pixels or Code? Benchmarking Instruction-Guided Visual Artifact Operation12 Oct 2026
- 2026Scalable Minimal-Change Learning for Controllable Image Editing5 Oct 2026
- 2026Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications28 Jul 2026
- 2026HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editing17 May 2026
- 2026Editor's Choice: Evaluating Abstract Intent in Image Editing through Atomic Entity Analysis14 May 2026
- 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editing10 Apr 2026
- 2026CAMEO: A Conditional and Quality-Aware Multi-Agent Image Editing Orchestrator3 Apr 2026
- 2026CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructions27 Mar 2026
- 2026Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Models16 Mar 2026
- 2026VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-On12 Mar 2026
- 2026VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents27 Jan 2026
- 2025I2I-Bench: A Comprehensive Benchmark Suite for Image-to-Image Editing Models4 Dec 2025
12 newest of 14 All 14 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2025ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies
- 2026MCIE: Multimodal LLM-Driven Complex Instruction Image Editing with Spatial Guidance
- 2026DLEBench: Evaluating Small-scale Object Editing Ability for Instruction-based Image Editing Model
- 2026How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing
- 2026PaintBench: Deterministic Evaluation of Precise Visual Editing
- 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing
- 2026GEditBench v2: A Human-Aligned Benchmark for General Image Editing
- 2026UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs