mlsanityData to 14 Oct 2026

MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark Emphasizing Modality Consistency

arXiv 2406.13219 ↗First posted 19 Jun 2024cs.CV cs.CLcited by 22

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

Abstract (arXiv)

Multimodal large language models (MLLMs) are prone to non-factual or outdated knowledge issues, which can manifest as misreading and misrecognition errors due to the complexity of multimodal knowledge. Previous benchmarks have not systematically analyzed the performance of editing methods in correcting these two error types. To better represent and correct these errors, we decompose multimodal knowledge into its visual and textual components. Different error types correspond to different editing formats, which edit distinct parts of the multimodal knowledge. We present MC-MKE, a fine-grained Multimodal Knowledge Editing benchmark emphasizing Modality Consistency. Our benchmark facilitates independent correction of misreading and misrecognition errors by editing the corresponding knowledge component. We evaluate four multimodal knowledge editing methods on MC-MKE, revealing their limitations, particularly in terms of modality consistency. Our work highlights the challenges posed by multimodal knowledge editing and motivates further research in developing effective techniques for this task.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 30 references that are in the map. First posted on arXiv 19 Jun 2024.

Builds on

15 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing18 Feb

Other references · 14

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  5. 2015VQA: Visual Question Answeringcited by 2,506
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  7. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  8. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  9. 2022Memory-Based Model Editing at Scalecited by 447
  10. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  11. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  12. 2020Modifying Memories in Transformer Modelscited by 224
2 more
  1. 2019Image Captioning: Transforming Objects into Wordscited by 171
  2. 2019MMKG: Multi-Modal Knowledge Graphscited by 105

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 7.

22 papers.

Sort:

By area:

  1. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  2. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  3. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  4. 2025MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledgecited by 23
  5. 2024Visual-Oriented Fine-Grained Knowledge Editing for MultiModal Large Language Modelscited by 9
  6. 2025Exploring and Evaluating Multimodal Knowledge Reasoning Consistency of Multimodal Large Language Modelscited by 8
  7. 2024Identifying Knowledge Editing Types in Large Language Modelscited by 7
  8. 2025MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Modelscited by 7
  9. 2025SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Modelscited by 5
  10. 2026MMKU-Bench: A Multimodal Update Benchmark for Diverse Visual Knowledgecited by 5
  11. 2025SNS-Bench-VL: Benchmarking Multimodal Large Language Models in Social Networking Servicescited by 2
  12. 2025Hybrid-DMKG: A Hybrid Reasoning Framework over Dynamic Multimodal Knowledge Graphs for Multimodal Multihop QA with Knowledge Editingcited by 2
10 more
  1. 2025Are Any-to-Any Models More Consistent Across Modality Transfers Than Specialists?cited by 1
  2. 2025Towards Meta-Cognitive Knowledge Editing for Multimodal LLMscited by 1
  3. 2026Generalizable Multimodal Large Language Model Editing via Invariant Trajectory Learningcited by 1
  4. 2026Uncovering Entity Identity Confusion in Multimodal Knowledge Editingcited by 1
  5. 2025ADS-Edit: A Multimodal Knowledge Editing Dataset for Autonomous Driving Systemscited by 0
  6. 2025Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Modelscited by 0
  7. 2026ReasonEdit: Editing Vision-Language Models using Human Reasoningcited by 0
  8. 2026CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMscited by 0
  9. 2026Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignmentcited by 0
  10. 2026Multimodal Knowledge Edit-Scoped Generalization for Online Recursive MLLM Editingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing
  2. 2026Towards Localized and Disentangled Knowledge Editing for Multimodal Large Language Models
  3. 2024Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration
  4. 2023Can We Edit Multimodal Large Language Models?
  5. 2025Benchmarking and Rethinking Knowledge Editing for Large Language Models
  6. 2026TokenSwap: Benchmarking and Reducing the Modality Gap in Multimodal LLMs
  7. 2023Knowledge Editing for Large Language Models: A Survey
  8. 2025KBE-DME: Dynamic Multimodal Evaluation via Knowledge Enhanced Benchmark Evolution