mlsanityData to 14 Oct 2026

MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models

arXiv 2508.05083 ↗First posted 7 Aug 2025cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Recent advances in multimodal large language models (MLLMs) have significantly improved medical AI, enabling it to unify the understanding of visual and textual information. However, as medical knowledge continues to evolve, it is critical to allow these models to efficiently update outdated or incorrect information without retraining from scratch. Although textual knowledge editing has been widely studied, there is still a lack of systematic benchmarks for multimodal medical knowledge editing involving image and text modalities. To fill this gap, we present MedMKEB, the first comprehensive benchmark designed to evaluate the reliability, generality, locality, portability, and robustness of knowledge editing in medical multimodal large language models. MedMKEB is built on a high-quality medical visual question-answering dataset and enriched with carefully constructed editing tasks, including counterfactual correction, semantic generalization, knowledge transfer, and adversarial robustness. We incorporate human expert validation to ensure the accuracy and reliability of the benchmark. Extensive single editing and sequential editing experiments on state-of-the-art general and medical MLLMs demonstrate the limitations of existing knowledge-based editing approaches in medicine, highlighting the need to develop specialized editing strategies. MedMKEB will serve as a standard benchmark to promote the development of trustworthy and efficient medical knowledge editing algorithms.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 31 references that are in the map. First posted on arXiv 7 Aug 2025.

Builds on

16 of 31 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale27 Jun
  2. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine14 May
  3. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM14 Feb
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 12

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2022Mass-Editing Memory in a Transformercited by 1,001
  3. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  4. 2021Editing Factual Knowledge in Language Modelscited by 553
  5. 2022Memory-Based Model Editing at Scalecited by 447
  6. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  7. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  8. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 109
  9. 2024Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answeringcited by 31
  10. 2024VLKEB: A Large Vision-Language Model Knowledge Editing Benchmarkcited by 30
  11. 2025MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledgecited by 23
  12. 2025Beyond Memorization: A Rigorous Evaluation Framework for Medical Knowledge Editingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Evaluating and Understanding Model Editing for Medical Vision Language Modelscited by 0
  2. 2026MedKIT: Evaluating Knowledge Integration and Generalization in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MultiMedEdit: A Scenario-Aware Benchmark for Evaluating Knowledge Editing in Medical VQA
  2. 2024PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imaging
  3. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
  4. 2024Towards a Multimodal Large Language Model with Pixel-Level Insight for Biomedicine
  5. 2026IMCBench: A benchmark for multimodal LLMs in Image-grounded Medical Conversations
  6. 2025SMMILE: An Expert-Driven Benchmark for Multimodal Medical In-Context Learning
  7. 2024MedViLaM: A multimodal large language model with advanced generalizability and explainability for medical data understanding and generation
  8. 2026MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images