# Leveraging the Structure of Medical Data for Improved Representation Learning

arXiv: https://arxiv.org/abs/2507.02987 · HTML: https://mlsanity.com/paper/2507.02987/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 1 Jul 2025.

## Builds on (most cited first, 12 of 12)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [Delving into Masked Autoencoders for Multi-Label Thorax Disease Classification](https://mlsanity.com/paper/2210.12843/) (2022)
- [Unity by Diversity: Improved Representation Learning in Multimodal VAEs](https://mlsanity.com/paper/2403.05300/) (2024)
- [MultiMed: Massively Multimodal and Multitask Medical Understanding](https://mlsanity.com/paper/2408.12682/) (2024)
- [Clinical Document Classification Using Labeled and Unlabeled Data Across Hospitals](https://mlsanity.com/paper/1812.00677/) (2018)
- [Weakly-Supervised Multimodal Learning on MIMIC-CXR](https://mlsanity.com/paper/2411.10356/) (2024)

## Built on by (most cited first, 2 of 2)

- [Structure is Supervision: Multiview Masked Autoencoders for Radiology](https://mlsanity.com/paper/2511.22294/) (2025)
- [Reading the Whole Heart: Latent-Attention Masked Autoencoders for Multimodal Cardiac Representation Learning](https://mlsanity.com/paper/2609.12035/) (2026)

## Abstract (arXiv)

Building generalizable medical AI systems requires pretraining strategies that are data-efficient and domain-aware. Unlike internet-scale corpora, clinical datasets such as MIMIC-CXR offer limited image counts and scarce annotations, but exhibit rich internal structure through multi-view imaging. We propose a self-supervised framework that leverages the inherent structure of medical datasets. Specifically, we treat paired chest X-rays (i.e., frontal and lateral views) as natural positive pairs, learning to reconstruct each view from sparse patches while aligning their latent embeddings. Our method requires no textual supervision and produces informative representations. Evaluated on MIMIC-CXR, we show strong performance compared to supervised objectives and baselines being trained without leveraging structure. This work provides a lightweight, modality-agnostic blueprint for domain-specific pretraining where data is structured but scarce