# Decipher-MR: A Vision-Language Foundation Model for 3D MRI Representations

arXiv: https://arxiv.org/abs/2509.21249 · HTML: https://mlsanity.com/paper/2509.21249/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 10 of its 48 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 10 of 10)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images](https://mlsanity.com/paper/2310.15161/) (2023)
- [Merlin: A Computed Tomography Vision-Language Foundation Model and Dataset](https://mlsanity.com/paper/2406.06512/) (2024)
- [Advancing Multimodal Medical Capabilities of Gemini](https://mlsanity.com/paper/2405.03162/) (2024)
- [MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging](https://mlsanity.com/paper/2410.06542/) (2024)
- [BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentation](https://mlsanity.com/paper/2406.10395/) (2024)
- [How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model](https://mlsanity.com/paper/2404.09957/) (2024)
- [Triad: Vision Foundation Model for 3D Magnetic Resonance Imaging](https://mlsanity.com/paper/2502.14064/) (2025)
- [Advancing human-centric AI for robust X-ray analysis through holistic self-supervised learning](https://mlsanity.com/paper/2405.01469/) (2024)

## Built on by (most cited first, 6 of 6)

- [Pan-FM: A Pan-Organ Foundation Model with Saliency-Guided Masking for Missing Robustness](https://mlsanity.com/paper/2605.07055/) (2026)
- [MRI2Rep: Autoregressive Structured Report Generation for 3D Liver MRI](https://mlsanity.com/paper/2606.25279/) (2026)
- [AGA3DNet: Anatomy-Guided Gaussian Priors with Multi-view xLSTM for 3D Brain MRI Subtype Classification](https://mlsanity.com/paper/2605.07142/) (2026)
- [Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI](https://mlsanity.com/paper/2608.09991/) (2026)
- [A4-MR: Abnormality-aware Anatomical Alignment for MR](https://mlsanity.com/paper/2610.13056/) (2026)
- [Benchmarking Self-Supervised 3D Medical Vision Foundation Models and Transferability Estimation](https://mlsanity.com/paper/2610.17573/) (2026)

## Abstract (arXiv)

Magnetic Resonance Imaging is a critical imaging modality in clinical diagnosis and research, yet its complexity and heterogeneity hinder scalable, generalizable machine learning. Although foundation models have revolutionized language and vision tasks, their application to MRI remains constrained by data scarcity and narrow anatomical focus. We present Decipher-MR, a 3D MRI-specific vision-language foundation model trained on 200,000 MRI series from over 22,000 studies spanning diverse anatomical regions, sequences, and pathologies. Decipher-MR integrates self-supervised vision learning with report-guided text supervision to build robust representations for broad applications. To enable efficient use, Decipher-MR supports a modular design that enables tuning of lightweight, task-specific decoders attached to a frozen pretrained encoder. Following this setting, we evaluate Decipher-MR across disease classification, demographic prediction, anatomical localization, and cross-modal retrieval, demonstrating consistent improvements over existing foundation models and task-specific approaches. These results position Decipher-MR as a versatile foundation for MRI-based AI in clinical and research settings.