mlsanityData to 14 Oct 2026

MedM-VL: What Makes a Good Medical LVLM?

arXiv 2504.04323 ↗First posted 6 Apr 2025cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Medical image analysis is essential in modern healthcare. Deep learning has redirected research focus toward complex medical multimodal tasks, including report generation and visual question answering. Traditional task-specific models often fall short in handling these challenges. Large vision-language models (LVLMs) offer new solutions for solving such tasks. In this study, we build on the popular LLaVA framework to systematically explore model architectures and training strategies for both 2D and 3D medical LVLMs. We present extensive empirical findings and practical guidance. To support reproducibility and future research, we release a modular codebase, MedM-VL, and two pre-trained models: MedM-VL-2D for 2D medical image analysis and MedM-VL-CT-Chest for 3D CT-based applications. The code is available at: https://github.com/MSIIP/MedM-VL

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 6 Apr 2025.

Builds on

33 of 48 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2023Qwen Technical Reportcited by 3,344
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  11. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  12. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 690
21 more
  1. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  2. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  3. 2017Deep Learning for Medical Image Analysiscited by 401
  4. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
  5. 2020A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promisescited by 231
  6. 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 199
  7. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 179
  8. 2024TinyLLaVA: A Framework of Small-scale Large Multimodal Modelscited by 164
  9. 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 162
  10. 2019Clinically Accurate Chest X-Ray Report Generationcited by 139
  11. 2023Large language models in healthcare and medical domain: A reviewcited by 127
  12. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 125
  13. 2024Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directionscited by 97
  14. 2023PromptMRG: Diagnosis-Driven Prompts for Medical Report Generationcited by 95
  15. 2024Advancing Multimodal Medical Capabilities of Geminicited by 90
  16. 2023Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicinecited by 83
  17. 2023SA-Med2D-20M Dataset: Segment Anything in 2D Medical Imaging with 20 Million maskscited by 55
  18. 2024A review of deep learning-based information fusion techniques for multimodal medical image classificationcited by 25
  19. 2024Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoEcited by 19
  20. 2024Med-2E3: A 2D-Enhanced 3D Medical Multimodal Large Language Modelcited by 15
  21. 2024TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generationcited by 6
  2. 2026U-VLM: Hierarchical Vision Language Modeling for Report Generationcited by 5
  3. 2025Enhancing Multi-task Learning Capability of Medical Generalist Foundation Model via Image-centric Multi-annotation Datacited by 2
  4. 2026Region-Grounded Report Generation for 3D Medical Imaging: A Fine-Grained Dataset and Graph-Enhanced Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Medical Large Vision Language Models with Multi-Image Visual Ability
  2. 2024On Large Visual Language Models for Medical Imaging Analysis: An Empirical Study
  3. 2025Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis
  4. 2025How Far Have Medical Vision-Language Models Come? A Comprehensive Benchmarking Study
  5. 2024Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology
  6. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review
  7. 2026Adapting 2D Multi-Modal Large Language Model for 3D CT Image Analysis
  8. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM