MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Lesion detection, symptom tracking, and visual explainability are central to real-world medical image analysis, yet current medical Vision-Language Models (VLMs) still lack mechanisms that translate their broad knowledge into clinically actionable outputs. To bridge this gap, we present MEDIC-AD, a clinically oriented VLM that strengthens these three capabilities through a stage-wise framework. First, learnable anomaly-aware tokens (<Ano>) encourage the model to focus on abnormal regions and build more discriminative lesion centered representations. Second, inter image difference tokens (<Diff>) explicitly encode temporal changes between studies, allowing the model to distinguish worsening, improvement, and stability in disease burden. Finally, a dedicated explainability stage trains the model to generate heatmaps that highlight lesion-related regions, offering clear visual evidence that is consistent with the model's reasoning. Through our staged design, MEDIC-AD steadily boosts performance across anomaly detection, symptom tracking, and anomaly segmentation, achieving state-of-the-art results compared with both closed source and medical-specialized baselines. Evaluations on real longitudinal clinical data collected from real hospital workflows further show that MEDIC-AD delivers stable predictions and clinically faithful explanations in practical patient-monitoring and decision-support workflows
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 66 references that are in the map. First posted on arXiv 28 Mar 2026.
Builds on
57 of 66 references.
Closest ancestors · 7
Same direction, newest first
- 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
- 2025MMXU: A Multi-Modal and Multi-X-ray Understanding Dataset for Disease Progression17 Feb
- 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine6 Aug
- 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM14 Feb
- 2024Pretraining Vision-Language Model for Difference Visual Question Answering in Longitudinal Chest X-rays14 Feb
- 2023Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering22 Jul
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun
Other references · 50
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2015VQA: Visual Question Answeringcited by 2,489
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2022Visual Prompt Tuningcited by 1,674
38 more
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
- 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
- 2014Show and Tell: A Neural Image Caption Generatorcited by 877
- 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
- 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
- 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
- 2021The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classificationcited by 507
- 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 504
- 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
- 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
- 2025MedGemma Technical Reportcited by 398
- 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 397
- 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
- 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
- 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
- 2023Towards Generalist Biomedical AIcited by 286
- 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 270
- 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 212
- 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
- 2020Can AI help in screening Viral and COVID-19 pneumonia?cited by 200
- 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
- 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Modelscited by 169
- 2024AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detectioncited by 126
- 2024GREEN: Generative Radiology Report Evaluation and Error Notationcited by 124
- 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Modelscited by 105
- 2023BMAD: Benchmarks for Medical Anomaly Detectioncited by 81
- 2022Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoningcited by 69
- 2024Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Imagescited by 69
- 2024RaTEScore: A Metric for Radiology Report Generationcited by 65
- 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Modelscited by 54
- 2021Describing and Localizing Multiple Changes with Transformerscited by 47
- 2021A Structure-Aware Relation Network for Thoracic Diseases Detection and Segmentationcited by 33
- 2024DEAL: Disentangle and Localize Concept-level Explanations for VLMscited by 14
- 2025MMPB: It's Time for Multi-Modal Personalizationcited by 14
- 2025Citrus-V: Advancing Medical Foundation Models with Unified Medical Image Grounding for Clinical Reasoningcited by 8
- 2025Grounding Chest X-Ray Visual Question Answering with Generated Radiology Reportscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Enhancing Abnormality Grounding for Vision Language Models with Knowledge Descriptions
- 2026Clinical Trajectory Alignment for Medical Vision-Language Pre-training
- 2024MediCLIP: Adapting CLIP for Few-shot Medical Image Anomaly Detection
- 2026MedCORE: Criteria-Grounded Clinical Reasoning for Interpretable Medical Image Diagnosis
- 2025Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback
- 2025SD-MAD: Sign-Driven Few-shot Multi-Anomaly Detection in Medical Images
- 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review
- 2025MedFILIP: Medical Fine-grained Language-Image Pre-training