# MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark Emphasizing Modality Consistency

arXiv: https://arxiv.org/abs/2406.13219 · HTML: https://mlsanity.com/paper/2406.13219/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 15 of 15)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [Otter: A Multi-Modal Model with In-Context Instruction Tuning](https://mlsanity.com/paper/2305.03726/) (2023)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Modifying Memories in Transformer Models](https://mlsanity.com/paper/2012.00363/) (2020)
- [Image Captioning: Transforming Objects into Words](https://mlsanity.com/paper/1906.05963/) (2019)
- [MMKG: Multi-Modal Knowledge Graphs](https://mlsanity.com/paper/1903.05485/) (2019)
- [MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing](https://mlsanity.com/paper/2402.14835/) (2024)

## Built on by (most cited first, 22 of 22)

- [A Survey on Evaluation of Multimodal Large Language Models](https://mlsanity.com/paper/2408.15769/) (2024)
- [Can Knowledge Editing Really Correct Hallucinations?](https://mlsanity.com/paper/2410.16251/) (2024)
- [A Survey on Multimodal Benchmarks: In the Era of Large AI Models](https://mlsanity.com/paper/2409.18142/) (2024)
- [MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge](https://mlsanity.com/paper/2502.19870/) (2025)
- [Visual-Oriented Fine-Grained Knowledge Editing for MultiModal Large Language Models](https://mlsanity.com/paper/2411.12790/) (2024)
- [Exploring and Evaluating Multimodal Knowledge Reasoning Consistency of Multimodal Large Language Models](https://mlsanity.com/paper/2503.04801/) (2025)
- [Identifying Knowledge Editing Types in Large Language Models](https://mlsanity.com/paper/2409.19663/) (2024)
- [MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models](https://mlsanity.com/paper/2510.19457/) (2025)
- [SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models](https://mlsanity.com/paper/2510.16917/) (2025)
- [MMKU-Bench: A Multimodal Update Benchmark for Diverse Visual Knowledge](https://mlsanity.com/paper/2603.15117/) (2026)
- [SNS-Bench-VL: Benchmarking Multimodal Large Language Models in Social Networking Services](https://mlsanity.com/paper/2505.23065/) (2025)
- [Hybrid-DMKG: A Hybrid Reasoning Framework over Dynamic Multimodal Knowledge Graphs for Multimodal Multihop QA with Knowledge Editing](https://mlsanity.com/paper/2512.00881/) (2025)
- [Are Any-to-Any Models More Consistent Across Modality Transfers Than Specialists?](https://mlsanity.com/paper/2505.24211/) (2025)
- [Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs](https://mlsanity.com/paper/2509.05714/) (2025)
- [Generalizable Multimodal Large Language Model Editing via Invariant Trajectory Learning](https://mlsanity.com/paper/2601.19700/) (2026)
- [Uncovering Entity Identity Confusion in Multimodal Knowledge Editing](https://mlsanity.com/paper/2605.06096/) (2026)
- [ADS-Edit: A Multimodal Knowledge Editing Dataset for Autonomous Driving Systems](https://mlsanity.com/paper/2503.20756/) (2025)
- [Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Models](https://mlsanity.com/paper/2506.11253/) (2025)
- [ReasonEdit: Editing Vision-Language Models using Human Reasoning](https://mlsanity.com/paper/2602.02408/) (2026)
- [CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs](https://mlsanity.com/paper/2605.06115/) (2026)
- [Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment](https://mlsanity.com/paper/2605.23780/) (2026)
- [Multimodal Knowledge Edit-Scoped Generalization for Online Recursive MLLM Editing](https://mlsanity.com/paper/2607.01978/) (2026)

## Abstract (arXiv)

Multimodal large language models (MLLMs) are prone to non-factual or outdated knowledge issues, which can manifest as misreading and misrecognition errors due to the complexity of multimodal knowledge. Previous benchmarks have not systematically analyzed the performance of editing methods in correcting these two error types. To better represent and correct these errors, we decompose multimodal knowledge into its visual and textual components. Different error types correspond to different editing formats, which edit distinct parts of the multimodal knowledge. We present MC-MKE, a fine-grained Multimodal Knowledge Editing benchmark emphasizing Modality Consistency. Our benchmark facilitates independent correction of misreading and misrecognition errors by editing the corresponding knowledge component. We evaluate four multimodal knowledge editing methods on MC-MKE, revealing their limitations, particularly in terms of modality consistency. Our work highlights the challenges posed by multimodal knowledge editing and motivates further research in developing effective techniques for this task.