# LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMs

arXiv: https://arxiv.org/abs/2507.16193 · HTML: https://mlsanity.com/paper/2507.16193/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 7 references that are in the map.
- First posted on arXiv 22 Jul 2025.

## Builds on (most cited first, 5 of 5)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [AIGCIQA2023: A Large-scale Image Quality Assessment Database for AI Generated Images: from the Perspectives of Quality, Authenticity and Correspondence](https://mlsanity.com/paper/2307.00211/) (2023)
- [Large Multi-modality Model Assisted AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2404.17762/) (2024)
- [Content-Variant Reference Image Quality Assessment via Knowledge Distillation](https://mlsanity.com/paper/2202.13123/) (2022)

## Built on by (most cited first, 24 of 24)

- [MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restoration](https://mlsanity.com/paper/2510.08508/) (2025)
- [EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing](https://mlsanity.com/paper/2512.11715/) (2025)
- [I2I-Bench: A Comprehensive Benchmark Suite for Image-to-Image Editing Models](https://mlsanity.com/paper/2512.04660/) (2025)
- [ManipShield: A Unified Framework for Image Manipulation Detection, Localization and Explanation](https://mlsanity.com/paper/2511.14259/) (2025)
- [Taming I2V models for Image HOI Editing: A Cognitive Benchmark and Agentic Self-Correcting Framework](https://mlsanity.com/paper/2606.19073/) (2026)
- [EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing](https://mlsanity.com/paper/2603.14916/) (2026)
- [VTONQA: A Multi-Dimensional Quality Assessment Dataset for Virtual Try-on](https://mlsanity.com/paper/2601.02945/) (2026)
- [Evaluating Image Editing with LLMs: A Comprehensive Benchmark and Intermediate-Layer Probing Approach](https://mlsanity.com/paper/2603.19775/) (2026)
- [CAMEO: A Conditional and Quality-Aware Multi-Agent Image Editing Orchestrator](https://mlsanity.com/paper/2604.03156/) (2026)
- [AutoEdit: Automatic Hyperparameter Tuning for Image Editing](https://mlsanity.com/paper/2509.15031/) (2025)
- [VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents](https://mlsanity.com/paper/2602.00122/) (2026)
- [MagicWand: A Universal Agent for Generation and Evaluation Aligned with User Preference](https://mlsanity.com/paper/2511.18352/) (2025)
- [Fine-Grained Human Pose Editing Assessment via Layer-Selective MLLMs](https://mlsanity.com/paper/2601.10369/) (2026)
- [Decoupling Perception and Calibration: Label-Efficient Image Quality Assessment Framework](https://mlsanity.com/paper/2601.20689/) (2026)
- [EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinement](https://mlsanity.com/paper/2605.07457/) (2026)
- [ReasonEdit: Towards Interpretable Image Editing Evaluation via Reinforcement Learning](https://mlsanity.com/paper/2605.07477/) (2026)
- [Towards Characterizing Scientific Image Utility and Upgradability](https://mlsanity.com/paper/2606.03401/) (2026)
- [MIEScore: Human-Aligned Evaluation for Multi-Source Image Editing](https://mlsanity.com/paper/2608.02059/) (2026)
- [ITIScore: An Image-to-Text-to-Image Rating Framework for the Image Captioning Ability of MLLMs](https://mlsanity.com/paper/2604.03765/) (2026)
- [Redefining Quality Criteria and Distance-Aware Score Modeling for Image Editing Assessment](https://mlsanity.com/paper/2604.12175/) (2026)
- [Measured Sliders: Learning Continuous Controls from Differentiable Image Measurements](https://mlsanity.com/paper/2609.05234/) (2026)
- [Do MLLM Judges Judge the Edit? Auditing Bias in Image Editing Evaluation with Verified Quality Preservation](https://mlsanity.com/paper/2610.01670/) (2026)
- [Beyond Plausibility: Verifiable Fine-Grained Image Editing on Structured Assets](https://mlsanity.com/paper/2610.04381/) (2026)
- [Pixels or Code? Benchmarking Instruction-Guided Visual Artifact Operation](https://mlsanity.com/paper/2610.16444/) (2026)

## Abstract (arXiv)

The rapid advancement of Text-guided Image Editing (TIE) enables image modifications through text prompts. However, current TIE models still struggle to balance image quality, editing alignment, and consistency with the original image, limiting their practical applications. Existing TIE evaluation benchmarks and metrics have limitations on scale or alignment with human perception. To this end, we introduce EBench-18K, the first large-scale image Editing Benchmark including 18K edited images with fine-grained human preference annotations for evaluating TIE. Specifically, EBench-18K includes 1,080 source images with corresponding editing prompts across 21 tasks, 18K+ edited images produced by 17 state-of-the-art TIE models, 55K+ mean opinion scores (MOSs) assessed from three evaluation dimensions, and 18K+ question-answering (QA) pairs. Based on EBench-18K, we employ outstanding LMMs to assess edited images, while the evaluation results, in turn, provide insights into assessing the alignment between the LMMs' understanding ability and human preferences. Then, we propose LMM4Edit, a LMM-based metric for evaluating image Editing models from perceptual quality, editing alignment, attribute preservation, and task-specific QA accuracy in an all-in-one manner. Extensive experiments show that LMM4Edit achieves outstanding performance and aligns well with human preference. Zero-shot validation on the other datasets also shows the generalization ability of our model. The dataset and code are available at https://github.com/IntMeGroup/LMM4Edit.