# textttComplex-Edit: CoT-Like Instruction Generation for Complexity-Controllable Image Editing Benchmark

arXiv: https://arxiv.org/abs/2504.13143 · HTML: https://mlsanity.com/paper/2504.13143/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 34 references that are in the map.
- First posted on arXiv 17 Apr 2025.

## Builds on (most cited first, 29 of 29)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing](https://mlsanity.com/paper/2306.10012/) (2023)
- [OmniGen: Unified Image Generation](https://mlsanity.com/paper/2409.11340/) (2024)
- [Emu Edit: Precise Image Editing via Recognition and Generation Tasks](https://mlsanity.com/paper/2311.10089/) (2023)
- [UltraEdit: Instruction-based Fine-Grained Image Editing at Scale](https://mlsanity.com/paper/2407.05282/) (2024)
- [Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning](https://mlsanity.com/paper/2405.00451/) (2024)
- [AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea](https://mlsanity.com/paper/2411.15738/) (2024)
- [Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps](https://mlsanity.com/paper/2501.09732/) (2025)
- [VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation](https://mlsanity.com/paper/2312.14867/) (2023)
- [Self-Evaluation Guided Beam Search for Reasoning](https://mlsanity.com/paper/2305.00633/) (2023)
- [HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing](https://mlsanity.com/paper/2404.09990/) (2024)
- [Guiding Instruction-based Image Editing via Multimodal Large Language Models](https://mlsanity.com/paper/2309.17102/) (2023)
- [SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models](https://mlsanity.com/paper/2312.06739/) (2023)
- [HIVE: Harnessing Human Feedback for Instructional Visual Editing](https://mlsanity.com/paper/2303.09618/) (2023)
- [SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer](https://mlsanity.com/paper/2501.18427/) (2025)
- [Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data](https://mlsanity.com/paper/2404.01413/) (2024)
- [OmniEdit: Building Image Editing Generalist Models Through Specialist Supervision](https://mlsanity.com/paper/2411.07199/) (2024)
- [Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step](https://mlsanity.com/paper/2501.13926/) (2025)
- [Calibrated Self-Rewarding Vision Language Models](https://mlsanity.com/paper/2405.14622/) (2024)
- [ImagenHub: Standardizing the evaluation of conditional image generation models](https://mlsanity.com/paper/2310.01596/) (2023)
- [SeedEdit: Align Image Re-Generation to Image Editing](https://mlsanity.com/paper/2411.06686/) (2024)
- [Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment](https://mlsanity.com/paper/2410.14148/) (2024)

## Built on by (most cited first, 27 of 27)

- [KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models](https://mlsanity.com/paper/2505.16707/) (2025)
- [GPT-IMAGE-EDIT-1.5M: A Million-Scale, GPT-Generated Image Dataset](https://mlsanity.com/paper/2507.21033/) (2025)
- [OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editing](https://mlsanity.com/paper/2512.07826/) (2025)
- [ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies](https://mlsanity.com/paper/2506.12830/) (2025)
- [CompBench: Benchmarking Complex Instruction-guided Image Editing](https://mlsanity.com/paper/2505.12200/) (2025)
- [EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing](https://mlsanity.com/paper/2512.11715/) (2025)
- [MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation](https://mlsanity.com/paper/2512.14691/) (2025)
- [Image Editing As Programs with Diffusion Models](https://mlsanity.com/paper/2506.04158/) (2025)
- [EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editing](https://mlsanity.com/paper/2509.13399/) (2025)
- [Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning](https://mlsanity.com/paper/2507.01908/) (2025)
- [TextEditBench: Evaluating Reasoning-aware Text Editing Beyond Rendering](https://mlsanity.com/paper/2512.16270/) (2025)
- [Generative Visual Chain-of-Thought for Image Editing](https://mlsanity.com/paper/2603.01893/) (2026)
- [CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructions](https://mlsanity.com/paper/2603.26174/) (2026)
- [Meta-CoT: Enhancing Granularity and Generalization in Image Editing](https://mlsanity.com/paper/2604.24625/) (2026)
- [VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents](https://mlsanity.com/paper/2602.00122/) (2026)
- [PaintBench: Deterministic Evaluation of Precise Visual Editing](https://mlsanity.com/paper/2606.00188/) (2026)
- [PerfGuard: A Performance-Aware Agent for Visual Content Generation](https://mlsanity.com/paper/2601.22571/) (2026)
- [InEdit-Bench: Benchmarking Intermediate Logical Pathways for Intelligent Image Editing Models](https://mlsanity.com/paper/2603.03657/) (2026)
- [MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching](https://mlsanity.com/paper/2606.01985/) (2026)
- [InstructMoLE: Instruction-Guided Mixture of Low-rank Experts for Multi-Conditional Image Generation](https://mlsanity.com/paper/2512.21788/) (2025)
- [A Unified and Controllable Framework for Layered Image Generation with Visual Effects](https://mlsanity.com/paper/2601.15507/) (2026)
- [Dress-ED: Instruction-Guided Editing for Virtual Try-On and Try-Off](https://mlsanity.com/paper/2603.22607/) (2026)
- [Towards Robust Sequential Decomposition for Complex Image Editing](https://mlsanity.com/paper/2605.09233/) (2026)
- [Editor's Choice: Evaluating Abstract Intent in Image Editing through Atomic Entity Analysis](https://mlsanity.com/paper/2605.14842/) (2026)
- [Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications](https://mlsanity.com/paper/2607.25642/) (2026)
- [Beyond Plausibility: Verifiable Fine-Grained Image Editing on Structured Assets](https://mlsanity.com/paper/2610.04381/) (2026)
- [Rubric-CEPR: Self-Evolving Image Editing via Reward-Verified Self-Distillation](https://mlsanity.com/paper/2610.12469/) (2026)

## Abstract (arXiv)

We introduce textttComplex-Edit, a comprehensive benchmark designed to systematically evaluate instruction-based image editing models across instructions of varying complexity. To develop this benchmark, we harness GPT-4o to automatically collect a diverse set of editing instructions at scale. Our approach follows a well-structured ``Chain-of-Edit'' pipeline: we first generate individual atomic editing tasks independently and then integrate them to form cohesive, complex instructions. Additionally, we introduce a suite of metrics to assess various aspects of editing performance, along with a VLM-based auto-evaluation pipeline that supports large-scale assessments. Our benchmark yields several notable insights: 1) Open-source models significantly underperform relative to proprietary, closed-source models, with the performance gap widening as instruction complexity increases; 2) Increased instructional complexity primarily impairs the models' ability to retain key elements from the input images and to preserve the overall aesthetic quality; 3) Decomposing a complex instruction into a sequence of atomic steps, executed in a step-by-step manner, substantially degrades performance across multiple metrics; 4) A straightforward Best-of-N selection strategy improves results for both direct editing and the step-by-step sequential approach; and 5) We observe a ``curse of synthetic data'': when synthetic data is involved in model training, the edited images from such models tend to appear increasingly synthetic as the complexity of the editing instructions rises -- a phenomenon that intriguingly also manifests in the latest GPT-4o outputs.