# Unifying Biomedical Vision-Language Expertise: Towards a Generalist Foundation Model via Multi-CLIP Knowledge Distillation

arXiv: https://arxiv.org/abs/2506.22567 · HTML: https://mlsanity.com/paper/2506.22567/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- First posted on arXiv 27 Jun 2025.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Built on by (most cited first, 8 of 8)

- [Capabilities of GPT-5 on Multimodal Medical Reasoning](https://mlsanity.com/paper/2508.08224/) (2025)
- [DINOv3 with Test-Time Training for Medical Image Registration](https://mlsanity.com/paper/2508.14809/) (2025)
- [Performance of GPT-5 in Brain Tumor MRI Reasoning](https://mlsanity.com/paper/2508.10865/) (2025)
- [Forging a Dynamic Memory: Retrieval-Guided Continual Learning for Generalist Medical Foundation Models](https://mlsanity.com/paper/2512.13072/) (2025)
- [Efficient Vision Mamba for MRI Super-Resolution via Hybrid Selective Scanning](https://mlsanity.com/paper/2512.19676/) (2025)
- [An Efficient 3D Latent Diffusion Model for T1-contrast Enhanced MRI Generation](https://mlsanity.com/paper/2509.24194/) (2025)
- [MedP-CLIP: Medical CLIP with Region-Aware Prompt Integration](https://mlsanity.com/paper/2604.11197/) (2026)
- [Med-RADIO: Reducing All Medical Domains Into One via Multi-Teacher Distillation](https://mlsanity.com/paper/2609.37682/) (2026)

## Abstract (arXiv)

CLIP models pretrained on natural images with billion-scale image-text pairs have demonstrated impressive capabilities in zero-shot classification, cross-modal retrieval, and open-ended visual answering. However, transferring this success to biomedicine is hindered by the scarcity of large-scale biomedical image-text corpora, the heterogeneity of image modalities, and fragmented data standards across institutions. These limitations hinder the development of a unified and generalizable biomedical foundation model trained from scratch. To overcome this, we introduce MMKD-CLIP, a generalist biomedical foundation model developed via Multiple Medical CLIP Knowledge Distillation. Rather than relying on billion-scale raw data, MMKD-CLIP distills knowledge from nine state-of-the-art domain-specific or generalist biomedical CLIP models, each pretrained on millions of biomedical image-text pairs. Our two-stage training pipeline first performs CLIP-style pretraining on over 2.9 million biomedical image-text pairs from 26 image modalities, followed by feature-level distillation using over 19.2 million feature pairs extracted from teacher models. We evaluate MMKD-CLIP on 58 diverse biomedical datasets, encompassing over 10.8 million biomedical images across nine image modalities. The evaluation spans six core task types: zero-shot classification, linear probing, cross-modal retrieval, visual question answering, survival prediction, and cancer diagnosis. MMKD-CLIP consistently outperforms all teacher models while demonstrating remarkable robustness and generalization across image domains and task settings. These results underscore that multi-teacher knowledge distillation is a scalable and effective paradigm for building high-performing biomedical foundation models under the practical constraints of real-world data availability.