mlsanityData to 14 Oct 2026

Med-Flamingo: a Multimodal Medical Few-shot Learner

arXiv 2307.15189 ↗First posted 27 Jul 2023cs.CV cs.AIcited by 392

Not on the map. Why?

Cited by 186 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Medicine, by its nature, is a multifaceted domain that requires the synthesis of information across various modalities. Medical generative vision-language models (VLMs) make a first step in this direction and promise many exciting clinical applications. However, existing models typically have to be fine-tuned on sizeable down-stream datasets, which poses a significant limitation as in many medical applications data is scarce, necessitating models that are capable of learning from few examples in real-time. Here we propose Med-Flamingo, a multimodal few-shot learner adapted to the medical domain. Based on OpenFlamingo-9B, we continue pre-training on paired and interleaved medical image-text data from publications and textbooks. Med-Flamingo unlocks few-shot generative medical visual question answering (VQA) abilities, which we evaluate on several datasets including a novel challenging open-ended VQA dataset of visual USMLE-style problems. Furthermore, we conduct the first human evaluation for generative medical VQA where physicians review the problems and blinded generations in an interactive app. Med-Flamingo improves performance in generative medical VQA by up to 20% in clinician's rating and firstly enables multimodal medical few-shot adaptations, such as rationale generation. We release our model, code, and evaluation app under https://github.com/snap-stanford/med-flamingo.

Key facts. Cited by 392 papers in the map (data through 14 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

24 of 34 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2017Billion-scale similarity search with GPUscited by 2,497
  10. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  11. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  12. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
12 more
  1. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  2. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  3. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  4. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 493
  5. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  6. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  7. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  8. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 211
  9. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  10. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  11. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  12. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 87

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 24; 2024: 103; 2025: 153; 2026: 112.

392 papers.

Sort:

By area:

  1. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
  2. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  3. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 230
  4. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scalecited by 200
  5. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  6. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 179
  7. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 163
  8. 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 162
  9. 2025Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Modelscited by 144
  10. 2024Exploring scalable medical image encoders beyond text supervisioncited by 132
  11. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 131
  12. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 125
38 more
  1. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 124
  2. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  3. 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationcited by 114
  4. 2025Baichuan-Omni-1.5 Technical Reportcited by 111
  5. 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agentcited by 109
  6. 2023Large Language Models as Analogical Reasonerscited by 107
  7. 2024CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formatscited by 106
  8. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  9. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
  10. 2024Towards Building Multilingual Language Model for Medicinecited by 92
  11. 2024Advancing Multimodal Medical Capabilities of Geminicited by 90
  12. 2023Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videoscited by 86
  13. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 85
  14. 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicinecited by 81
  15. 2023Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosiscited by 77
  16. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  17. 2023Foundational Models in Medical Imaging: A Comprehensive Survey and Future Visioncited by 76
  18. 2023MAIRA-1: A specialised large multimodal model for radiology report generationcited by 74
  19. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 73
  20. 2025MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflowcited by 73
  21. 2024RegionGPT: Towards Region Understanding Vision Language Modelcited by 71
  22. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directionscited by 68
  23. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
  24. 2023Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcarecited by 62
  25. 2024RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Modelscited by 60
  26. 2024ROCOv2: Radiology Objects in COntext Version 2, an Updated Multimodal Image Datasetcited by 54
  27. 2024BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experimentscited by 53
  28. 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoderscited by 50
  29. 2024VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledgecited by 50
  30. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Modelscited by 46
  31. 2024MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imagingcited by 46
  32. 2025Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysiscited by 46
  33. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  34. 2024Membership Inference Attacks Against In-Context Learningcited by 43
  35. 2024RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysiscited by 42
  36. 2025Improving Medical Large Vision-Language Models with Abnormal-Aware Feedbackcited by 42
  37. 2023CLIP in Medical Imaging: A Surveycited by 40
  38. 2025Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detectorcited by 40

50 most cited of 392 All 392 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Advancing High Resolution Vision-Language Models in Biomedicine
  2. 2024MISS: A Generative Pretraining and Finetuning Approach for Med-VQA
  3. 2025Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQA
  4. 2025Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMs
  5. 2023Visual Question Answering in the Medical Domain
  6. 2025A Lightweight Large Vision-language Model for Multimodal Medical Images
  7. 2024STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Question-Answering
  8. 2024LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasound