mlsanityData to 7 Oct 2026

MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence

arXiv 2603.27176 ↗First posted 28 Mar 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Lesion detection, symptom tracking, and visual explainability are central to real-world medical image analysis, yet current medical Vision-Language Models (VLMs) still lack mechanisms that translate their broad knowledge into clinically actionable outputs. To bridge this gap, we present MEDIC-AD, a clinically oriented VLM that strengthens these three capabilities through a stage-wise framework. First, learnable anomaly-aware tokens (<Ano>) encourage the model to focus on abnormal regions and build more discriminative lesion centered representations. Second, inter image difference tokens (<Diff>) explicitly encode temporal changes between studies, allowing the model to distinguish worsening, improvement, and stability in disease burden. Finally, a dedicated explainability stage trains the model to generate heatmaps that highlight lesion-related regions, offering clear visual evidence that is consistent with the model's reasoning. Through our staged design, MEDIC-AD steadily boosts performance across anomaly detection, symptom tracking, and anomaly segmentation, achieving state-of-the-art results compared with both closed source and medical-specialized baselines. Evaluations on real longitudinal clinical data collected from real hospital workflows further show that MEDIC-AD delivers stable predictions and clinically faithful explanations in practical patient-monitoring and decision-support workflows

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 66 references that are in the map. First posted on arXiv 28 Mar 2026.

Builds on

57 of 66 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
  2. 2025MMXU: A Multi-Modal and Multi-X-ray Understanding Dataset for Disease Progression17 Feb
  3. 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine6 Aug
  4. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM14 Feb
  5. 2024Pretraining Vision-Language Model for Difference Visual Question Answering in Longitudinal Chest X-rays14 Feb
  6. 2023Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering22 Jul
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 50

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  7. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  10. 2015VQA: Visual Question Answeringcited by 2,489
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  12. 2022Visual Prompt Tuningcited by 1,674
38 more
  1. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  3. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  4. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  5. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  6. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  7. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
  8. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  9. 2021The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classificationcited by 507
  10. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 504
  11. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  12. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
  13. 2025MedGemma Technical Reportcited by 398
  14. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 397
  15. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  16. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  17. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  18. 2023Towards Generalist Biomedical AIcited by 286
  19. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 270
  20. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 212
  21. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  22. 2020Can AI help in screening Viral and COVID-19 pneumonia?cited by 200
  23. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
  24. 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Modelscited by 169
  25. 2024AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detectioncited by 126
  26. 2024GREEN: Generative Radiology Report Evaluation and Error Notationcited by 124
  27. 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Modelscited by 105
  28. 2023BMAD: Benchmarks for Medical Anomaly Detectioncited by 81
  29. 2022Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoningcited by 69
  30. 2024Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Imagescited by 69
  31. 2024RaTEScore: A Metric for Radiology Report Generationcited by 65
  32. 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Modelscited by 54
  33. 2021Describing and Localizing Multiple Changes with Transformerscited by 47
  34. 2021A Structure-Aware Relation Network for Thoracic Diseases Detection and Segmentationcited by 33
  35. 2024DEAL: Disentangle and Localize Concept-level Explanations for VLMscited by 14
  36. 2025MMPB: It's Time for Multi-Modal Personalizationcited by 14
  37. 2025Citrus-V: Advancing Medical Foundation Models with Unified Medical Image Grounding for Clinical Reasoningcited by 8
  38. 2025Grounding Chest X-Ray Visual Question Answering with Generated Radiology Reportscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Abnormality Grounding for Vision Language Models with Knowledge Descriptions
  2. 2026Clinical Trajectory Alignment for Medical Vision-Language Pre-training
  3. 2024MediCLIP: Adapting CLIP for Few-shot Medical Image Anomaly Detection
  4. 2026MedCORE: Criteria-Grounded Clinical Reasoning for Interpretable Medical Image Diagnosis
  5. 2025Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback
  6. 2025SD-MAD: Sign-Driven Few-shot Multi-Anomaly Detection in Medical Images
  7. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review
  8. 2025MedFILIP: Medical Fine-grained Language-Image Pre-training