mlsanityData to 14 Oct 2026

How Well Can General Vision-Language Models Learn Medicine By Watching Public Educational Videos?

arXiv 2504.14391 ↗First posted 19 Apr 2025cs.CVcited by 6

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Publicly available biomedical videos, such as those on YouTube, serve as valuable educational resources for medical students. Unlike standard machine learning datasets, these videos are designed for human learners, often mixing medical imagery with narration, explanatory diagrams, and contextual framing. In this work, we investigate whether such pedagogically rich, yet non-standardized and heterogeneous videos can effectively teach general-domain vision-language models biomedical knowledge. To this end, we introduce OpenBiomedVi, a biomedical video instruction tuning dataset comprising 1031 hours of video-caption and Q/A pairs, curated through a multi-step human-in-the-loop pipeline. Diverse biomedical video datasets are rare, and OpenBiomedVid fills an important gap by providing instruction-style supervision grounded in real-world educational content. Surprisingly, despite the informal and heterogeneous nature of these videos, the fine-tuned Qwen-2-VL models exhibit substantial performance improvements across most benchmarks. The 2B model achieves gains of 98.7% on video tasks, 71.2% on image tasks, and 0.2% on text tasks. The 7B model shows improvements of 37.09% on video and 11.2% on image tasks, with a slight degradation of 2.7% on text tasks compared to their respective base models. To address the lack of standardized biomedical video evaluation datasets, we also introduce two new expert curated benchmarks, MIMICEchoQA and SurgeryVideoQA. On these benchmarks, the 2B model achieves gains of 99.1% and 98.1%, while the 7B model shows gains of 22.5% and 52.1%, respectively, demonstrating the models' ability to generalize and perform biomedical video understanding on cleaner and more standardized datasets than those seen during training. These results suggest that educational videos created for human learning offer a surprisingly effective training signal for biomedical VLMs.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 65 references that are in the map. First posted on arXiv 19 Apr 2025.

Builds on

43 of 65 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine6 Aug
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 41

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  10. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
29 more
  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  2. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  3. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  4. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  5. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  6. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  8. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  9. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  10. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  11. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  12. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
  13. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  14. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  15. 2023Towards Generalist Biomedical AIcited by 289
  16. 2024Capabilities of Gemini Models in Medicinecited by 237
  17. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
  18. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  19. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 195
  20. 2023Quilt-1M: One Million Image-Text Pairs for Histopathologycited by 194
  21. 2024Apollo: An Exploration of Video Understanding in Large Multimodal Modelscited by 89
  22. 2024Fewer Truncations Improve Language Modelingcited by 35
  23. 2024Small Language Models Learn Enhanced Reasoning Skills from Medical Textbookscited by 30
  24. 2022A Dataset for Medical Instructional Video Classification and Question Answeringcited by 26
  25. 2023CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detectioncited by 23
  26. 2023NurViD: A Large Expert-Level Video Database for Nursing Procedure Activity Understandingcited by 20
  27. 2024EchoPrime: A Multi-Video View-Informed Vision-Language Model for Comprehensive Echocardiography Interpretationcited by 18
  28. 2023From Beginner to Expert: Modeling Medical Knowledge into General LLMscited by 14
  29. 2025MedicalNarratives: Connecting Medical Vision and Language with Localized Narrativescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning
  2. 2026OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
  3. 2025How Far Have Medical Vision-Language Models Come? A Comprehensive Benchmarking Study
  4. 2026OpenMedQ: Broad Open Pretraining for Medical Vision-Language Models
  5. 2025BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature
  6. 2025OmniV-Med: Scaling Medical Vision-Language Model for Universal Visual Understanding
  7. 2024Biomedical Visual Instruction Tuning with Clinician Preference Alignment
  8. 2024Advancing High Resolution Vision-Language Models in Biomedicine