# Towards Cardiac MRI Foundation Models: Comprehensive Visual-Tabular Representations for Whole-Heart Assessment and Beyond

arXiv: https://arxiv.org/abs/2504.13037 · HTML: https://mlsanity.com/paper/2504.13037/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 58 references that are in the map.
- First posted on arXiv 17 Apr 2025.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Scaling Language-Image Pre-training via Masking](https://mlsanity.com/paper/2212.00794/) (2022)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks](https://mlsanity.com/paper/2305.17100/) (2023)
- [Deep learning for cardiac image segmentation: A review](https://mlsanity.com/paper/1911.03723/) (2019)
- [Self Pre-training with Masked Autoencoders for Medical Image Classification and Segmentation](https://mlsanity.com/paper/2203.05573/) (2022)
- [Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data](https://mlsanity.com/paper/2303.14080/) (2023)
- [Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences](https://mlsanity.com/paper/1806.04066/) (2018)
- [TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Data](https://mlsanity.com/paper/2407.07582/) (2024)
- [NISF: Neural Implicit Segmentation Functions](https://mlsanity.com/paper/2309.08643/) (2023)
- [Learning Shape Priors for Robust Cardiac MR Segmentation from Multi-view Images](https://mlsanity.com/paper/1907.09983/) (2019)
- [MulViMotion: Shape-aware 3D Myocardial Motion Tracking from Multi-View Cardiac MRI](https://mlsanity.com/paper/2208.00034/) (2022)
- [Unlocking the diagnostic potential of electrocardiograms through information transfer from cardiac magnetic resonance imaging](https://mlsanity.com/paper/2308.05764/) (2023)
- [Towards a vision foundation model for comprehensive assessment of Cardiac MRI](https://mlsanity.com/paper/2410.01665/) (2024)
- [Joint Motion Correction and Super Resolution for Cardiac Segmentation via Latent Optimisation](https://mlsanity.com/paper/2107.03887/) (2021)
- [Learning-based and unrolled motion-compensated reconstruction for cardiac MR CINE imaging](https://mlsanity.com/paper/2209.03671/) (2022)
- [Whole Heart 3D+T Representation Learning Through Sparse 2D Cardiac MR Images](https://mlsanity.com/paper/2406.00329/) (2024)
- [Motion-compensated MR CINE reconstruction with reconstruction-driven motion estimation](https://mlsanity.com/paper/2302.02504/) (2023)
- [A Generalizable Deep Learning System for Cardiac MRI](https://mlsanity.com/paper/2312.00357/) (2023)
- [Classification, Regression and Segmentation directly from k-Space in Cardiac MRI](https://mlsanity.com/paper/2407.20108/) (2024)
- [A Multimodal Deep Learning Model for Cardiac Resynchronisation Therapy Response Prediction](https://mlsanity.com/paper/2107.10662/) (2021)
- [Spatio-temporal neural distance fields for conditional generative modeling of the heart](https://mlsanity.com/paper/2407.10663/) (2024)
- [Direct Cardiac Segmentation from Undersampled K-space Using Transformers](https://mlsanity.com/paper/2406.00192/) (2024)
- [Multimodal Representation Learning of Cardiovascular Magnetic Resonance Imaging](https://mlsanity.com/paper/2304.07675/) (2023)

## Built on by (most cited first, 14 of 14)

- [A versatile foundation model for cine cardiac magnetic resonance image analysis tasks](https://mlsanity.com/paper/2506.00679/) (2025)
- [Does DINOv3 Set a New Medical Vision Standard? Benchmarking 2D and 3D Classification, Segmentation, and Registration](https://mlsanity.com/paper/2509.06467/) (2025)
- [Unleashing the Power of Image-Tabular Self-Supervised Learning via Breaking Cross-Tabular Barriers](https://mlsanity.com/paper/2512.14026/) (2025)
- [Latent Interpolation Learning Using Diffusion Models for Cardiac Volume Reconstruction](https://mlsanity.com/paper/2508.13826/) (2025)
- [M3Ret: Unleashing Zero-shot Multimodal Medical Image Retrieval via Self-Supervision](https://mlsanity.com/paper/2509.01360/) (2025)
- [Enabling Ultra-Fast Cardiovascular Imaging Across Heterogeneous Clinical Environments with A Generalist Foundation Model and Multimodal Database](https://mlsanity.com/paper/2512.21652/) (2025)
- [No Image, No Problem: End-to-End Multi-Task Cardiac Analysis from Undersampled k-Space](https://mlsanity.com/paper/2603.09945/) (2026)
- [Med-DualLoRA: Local Adaptation of Foundation Models for 3D Cardiac MRI](https://mlsanity.com/paper/2603.10967/) (2026)
- [Opportunistic Cardiac Health Assessment: Estimating Phenotypes from Localizer MRI through Multi-Modal Representations](https://mlsanity.com/paper/2603.13590/) (2026)
- [MARCUS: An agentic, multimodal vision-language model for cardiac diagnosis and management](https://mlsanity.com/paper/2603.22179/) (2026)
- [Learning from Acquisition: Metadata-driven Multimodal Pre-training for Cardiac MRI](https://mlsanity.com/paper/2606.28991/) (2026)
- [Foundation Models Adaptation for Multi-View Multi-modal Cardiac MRI Segmentation and Direct Ejection Fraction Estimation](https://mlsanity.com/paper/2608.07291/) (2026)
- [ORION-CMR: On-scanner Reporting with Integrated Foundation Model for End-to-End Cardiac MRI Analysis and Interpretation](https://mlsanity.com/paper/2609.23950/) (2026)
- [MMAP: Multimodal Missing-Aware Pretraining for Longitudinal Alzheimer's Prediction](https://mlsanity.com/paper/2609.26617/) (2026)

## Abstract (arXiv)

Cardiac magnetic resonance imaging is the gold standard for non-invasive cardiac assessment, offering rich spatio-temporal views of the cardiac anatomy and physiology. Patient-level health factors, such as demographics, metabolic, and lifestyle, are known to substantially influence cardiovascular health and disease risk, yet remain uncaptured by CMR alone. To holistically understand cardiac health and to enable the best possible interpretation of an individual's disease risk, CMR and patient-level factors must be jointly exploited within an integrated framework. Recent multi-modal approaches have begun to bridge this gap, yet they often rely on limited spatio-temporal data and focus on isolated clinical tasks, thereby hindering the development of a comprehensive representation for cardiac health evaluation. To overcome these limitations, we introduce ViTa, a step toward foundation models that delivers a comprehensive representation of the heart and a precise interpretation of individual disease risk. Leveraging data from 42,000 UK Biobank participants, ViTa integrates 3D+T cine stacks from short-axis and long-axis views, enabling a complete capture of the cardiac cycle. These imaging data are then fused with detailed tabular patient-level factors, enabling context-aware insights. This multi-modal paradigm supports a wide spectrum of downstream tasks, including cardiac phenotype and physiological feature prediction, segmentation, and classification of cardiac and metabolic diseases within a single unified framework. By learning a shared latent representation that bridges rich imaging features and patient context, ViTa moves beyond traditional, task-specific models toward a universal, patient-specific understanding of cardiac health, highlighting its potential to advance clinical utility and scalability in cardiac analysis.