mlsanityData to 7 Oct 2026

Enhancing Medical Large Vision-Language Models via Alignment Distillation

arXiv 2512.18554 ↗First posted 21 Dec 2025cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Medical Large Vision-Language Models (Med-LVLMs) have shown promising results in clinical applications, but often suffer from hallucinated outputs due to misaligned visual understanding. In this work, we identify two fundamental limitations contributing to this issue: insufficient visual representation learning and poor visual attention alignment. To address these problems, we propose MEDALIGN, a simple, lightweight alignment distillation framework that transfers visual alignment knowledge from a domain-specific Contrastive Language-Image Pre-training (CLIP) model to Med-LVLMs. MEDALIGN introduces two distillation losses: a spatial-aware visual alignment loss based on visual token-level similarity structures, and an attention-aware distillation loss that guides attention toward diagnostically relevant regions. Extensive experiments on medical report generation and medical visual question answering (VQA) benchmarks show that MEDALIGN consistently improves both performance and interpretability, yielding more visually grounded outputs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 40 references that are in the map. First posted on arXiv 21 Dec 2025.

Builds on

28 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Focus on What Matters: Enhancing Medical Vision-Language Models with Automatic Attention Alignment Tuning24 May
  2. 2024UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities13 Dec
  3. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM14 Feb
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 24

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2023Vision Transformers Need Registerscited by 844
  5. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  6. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  7. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  8. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 468
  9. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  10. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  11. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  12. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
12 more
  1. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  2. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 196
  3. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 174
  4. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 139
  5. 2023Word Embeddings Are Steers for Language Modelscited by 70
  6. 2024DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucinationcited by 47
  7. 2024CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generationcited by 25
  8. 2025Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMscited by 23
  9. 2024MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Contextcited by 19
  10. 2025Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigationcited by 9
  11. 2024Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decodingcited by 8
  12. 2024HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decodingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MedSIGHT: Towards Grounded Visual Comprehension in Medical Large Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning
  2. 2025Alignment, Mining and Fusion: Representation Alignment with Hard Negative Mining and Selective Knowledge Fusion for Medical Visual Question Answering
  3. 2025Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models
  4. 2025More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era
  5. 2025HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language Models
  6. 2026SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation
  7. 2026Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs
  8. 2025QwenCLIP: Boosting Medical Vision-Language Pretraining via LLM Embeddings and Prompt tuning