# Advancing Medical Representation Learning Through High-Quality Data

arXiv: https://arxiv.org/abs/2503.14377 · HTML: https://mlsanity.com/paper/2503.14377/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents](https://mlsanity.com/paper/2303.07240/) (2023)
- [Quilt-1M: One Million Image-Text Pairs for Histopathology](https://mlsanity.com/paper/2306.11207/) (2023)
- [Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP](https://mlsanity.com/paper/2208.05516/) (2022)
- [Medical Visual Question Answering: A Survey](https://mlsanity.com/paper/2111.10056/) (2021)
- [MedICaT: A Dataset of Medical Images, Captions, and Textual References](https://mlsanity.com/paper/2010.06000/) (2020)
- [DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation](https://mlsanity.com/paper/2011.00569/) (2020)
- [BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature](https://mlsanity.com/paper/2501.07171/) (2025)
- [EXSCLAIM! -- An automated pipeline for the construction of labeled materials imaging datasets from literature](https://mlsanity.com/paper/2103.10631/) (2021)
- [Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning](https://mlsanity.com/paper/2406.07450/) (2024)

## Built on by (most cited first, 8 of 8)

- [Open-PMC-18M: A High-Fidelity Large Scale Medical Dataset for Multimodal Representation Learning](https://mlsanity.com/paper/2506.02738/) (2025)
- [When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains](https://mlsanity.com/paper/2603.01301/) (2026)
- [Multi-Aspect Knowledge-Enhanced Medical Vision-Language Pretraining with Multi-Agent Data Generation](https://mlsanity.com/paper/2512.03445/) (2025)
- [Forging a Dynamic Memory: Retrieval-Guided Continual Learning for Generalist Medical Foundation Models](https://mlsanity.com/paper/2512.13072/) (2025)
- [Does medical specialization of VLMs enhance discriminative power?: A comprehensive investigation through feature distribution analysis](https://mlsanity.com/paper/2601.14774/) (2026)
- [Scientific Domain Knowledge Improves Vision-Language Fundus Models](https://mlsanity.com/paper/2605.02720/) (2026)
- [IRIS: An Intelligent Vision-Language System for Ocular Surface Diseases via Topic Tree and Scene-Driven VQA Generation](https://mlsanity.com/paper/2607.04344/) (2026)
- [Pretraining of Medical Visual Encoders Toward Multi-modal Large Language Models](https://mlsanity.com/paper/2609.23860/) (2026)

## Abstract (arXiv)

Despite the growing scale of medical Vision-Language datasets, the impact of dataset quality on model performance remains under-explored. We introduce Open-PMC, a high-quality medical dataset from PubMed Central, containing 2.2 million image-text pairs, enriched with image modality annotations, subfigures, and summarized in-text references. Notably, the in-text references provide richer medical context, extending beyond the abstract information typically found in captions. Through extensive experiments, we benchmark Open-PMC against larger datasets across retrieval and zero-shot classification tasks. Our results show that dataset quality-not just size-drives significant performance gains. We complement our benchmark with an in-depth analysis of feature representation. Our findings highlight the crucial role of data curation quality in advancing multimodal medical AI. We release Open-PMC, along with the trained models and our codebase.