mlsanityData to 7 Oct 2026

ClinKD: Cross-Modal Clinical Knowledge Distiller For Multi-Task Medical Images

arXiv 2502.05928 ↗First posted 9 Feb 2025cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Medical Visual Question Answering (Med-VQA) represents a critical and challenging subtask within the general VQA domain. Despite significant advancements in general VQA, multimodal large language models (MLLMs) still exhibit substantial limitations when handling multi-task VQA scenarios. These limitations manifest through erroneous spatial localization and misinterpretation of medical images, which primarily arise from two fundamental issues: inadequate image-text alignment and insufficient domain-specified knowledge for medical applications. To address these issues, we introduce the Cross-Modal Clinical Knowledge Distiller (ClinKD), an innovative framework designed to enhance image-text alignment and establish more effective medical knowledge transformation mechanisms, which enables MLLMs to perform better even when lacking prior medical knowledge. Our extensive experimental evaluations demonstrate that the ClinKD achieves state-of-the-art performance on several datasets which are challenging for Med-VQA task. The results indicate that our approach not only significantly improves image-text alignment but also effectively enables MLLMs to adapt to the medical knowledge. The source code for ClinKD is available at: https://github.com/overloadedHenry/ClinKD.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 42 of its 50 references that are in the map. First posted on arXiv 9 Feb 2025.

Builds on

42 of 50 references.

References · 42

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  8. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  9. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  10. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  11. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  12. 2019Similarity-Preserving Knowledge Distillationcited by 507
30 more
  1. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  2. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  3. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  4. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  5. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  6. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 247
  7. 2024Rotary Position Embedding for Vision Transformercited by 226
  8. 2018Learning Deep Representations with Probabilistic Knowledge Transfercited by 209
  9. 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Taskscited by 201
  10. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 134
  11. 2024Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Modelscited by 97
  12. 2021Medical Visual Question Answering: A Surveycited by 93
  13. 2025VideoRoPE: What Makes for Good Video Rotary Position Embedding?cited by 63
  14. 2023SA-Med2D-20M Dataset: Segment Anything in 2D Medical Imaging with 20 Million maskscited by 54
  15. 2022Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Videocited by 43
  16. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Modelscited by 31
  17. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Taskscited by 27
  18. 2021ITA: Image-Text Alignments for Multi-Modal Named Entity Recognitioncited by 24
  19. 2024VkD: Improving Knowledge Distillation using Orthogonal Projectionscited by 24
  20. 2024R-LLaVA: Improving Med-VQA Understanding through Visual Region of Interestcited by 21
  21. 2023Vision-Language Instruction Tuning: A Review and Analysiscited by 17
  22. 2023KD-DLGAN: Data Limited Image Generation via Knowledge Distillationcited by 16
  23. 2024A Refer-and-Ground Multimodal Large Language Model for Biomedicinecited by 14
  24. 2023Mixed Pseudo Labels for Semi-Supervised Object Detectioncited by 10
  25. 2024Parameter-Efficient Fine-Tuning Medical Multimodal Large Language Models for Medical Visual Groundingcited by 9
  26. 2024Visual-Oriented Fine-Grained Knowledge Editing for MultiModal Large Language Modelscited by 9
  27. 2024HC-LLM: Historical-Constrained Large Language Models for Radiology Report Generationcited by 9
  28. 2025MedFILIP: Medical Fine-grained Language-Image Pre-trainingcited by 8
  29. 2024MedVisionLlama: Leveraging Pre-Trained Large Language Model Layers to Enhance Medical Image Segmentationcited by 4
  30. 2022Consecutive Knowledge Meta-Adaptation Learning for Unsupervised Medical Diagnosiscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Datacited by 5
  2. 2026FADA: Accessible fetal ultrasound interpretation and annotation with a selectively distilled unified vision-language modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM
  2. 2026KG-CMI: Knowledge graph enhanced cross-Mamba interaction for medical visual question answering
  3. 2025Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQA
  4. 2024Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answering
  5. 2025MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation
  6. 2025Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning
  7. 2026From Reasoning to Pixels: Grounded Medical Multimodal LLMs for VQA and Segmentation
  8. 2024MC-CoT: A Modular Collaborative CoT Framework for Zero-shot Medical-VQA with LLM and MLLM Integration