mlsanityData to 7 Oct 2026

MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicine

arXiv 2603.00842 ↗First posted 1 Mar 2026cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Biomedical multimodal assistants have the potential to unify radiology, pathology, and clinical-text reasoning, yet a critical deployment gap remains: top-performing systems are either closed-source or computationally prohibitive, precluding the on-premises deployment required for patient privacy and PHI compliance. We introduce MEDGPT-OSS, an open-weight, 20B-parameter generalist vision-language model designed to facilitate open research in clinical AI. Rather than relying on architectural complexity, MEDGPT-OSS pairs the GPT-oss language backbone with a visual front-end via a optimized, three-stage training curriculum. By progressively domain-adapting these modules through rigorous data curation and long-context multimodal alignment, we demonstrate that a 20B model can bridge the capacity gap. It successfully outperforms larger open medical models on out-of-distribution (OOD) multimodal reasoning and complex text-only clinical tasks. By unifying diverse modalities under a single instruction-following interface, MEDGPT-OSS maintains a parameter-efficient footprint fully compatible with commodity GPUs. We release the complete training recipe, open-weight checkpoints, and a rigorous evaluation harness to serve as a verifiable foundation for privacy-preserving, institution-specific clinical AI research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 66 references that are in the map. First posted on arXiv 1 Mar 2026.

Builds on

50 of 66 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
  2. 2025QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training31 May
  3. 2025Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning23 May
  4. 2025BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature13 Jan
  5. 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine6 Aug
  6. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun
  7. 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks26 May

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2023Qwen Technical Reportcited by 3,273
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  11. 2025Qwen3-VL Technical Reportcited by 2,534
  12. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
31 more
  1. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  2. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  3. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  4. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
  5. 2022TotalSegmentator: robust segmentation of 104 anatomical structures in CT imagescited by 597
  6. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  7. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  8. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  9. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  10. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  11. 2025MedGemma Technical Reportcited by 398
  12. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
  13. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  14. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  15. 2023Towards Generalist Biomedical AIcited by 286
  16. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  17. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
  18. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  19. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 212
  20. 2023Quilt-1M: One Million Image-Text Pairs for Histopathologycited by 192
  21. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 149
  22. 2024Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questionscited by 106
  23. 2025Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understandingcited by 96
  24. 2023TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbonescited by 77
  25. 2024RaTEScore: A Metric for Radiology Report Generationcited by 65
  26. 2024ROCOv2: Radiology Objects in COntext Version 2, an Updated Multimodal Image Datasetcited by 54
  27. 2025MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoningcited by 33
  28. 2024Libra: Leveraging Temporal Images for Biomedical Radiology Analysiscited by 29
  29. 2025OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoningcited by 10
  30. 2025RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflowscited by 8
  31. 2025EditGRPO: Reinforcement Learning with Post-Rollout Edits for Clinically Accurate Chest X-Ray Report Generationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MMedExpert-R1: Strengthening Multimodal Medical Reasoning via Domain-Specific Adaptation and Clinical Guideline Reinforcement
  2. 2026MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images
  3. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
  4. 2025Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models
  5. 2026MEDVISTAGYM: A Scalable Training Environment for Thinking with Medical Images via Tool-Integrated Reinforcement Learning
  6. 2024MedViLaM: A multimodal large language model with advanced generalizability and explainability for medical data understanding and generation
  7. 2026OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
  8. 2025MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text