# Foundation Model for Whole-Heart Segmentation: Leveraging Student-Teacher Learning in Multi-Modal Medical Imaging

arXiv: https://arxiv.org/abs/2503.19005 · HTML: https://mlsanity.com/paper/2503.19005/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 80 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Builds on (most cited first, 30 of 35)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation](https://mlsanity.com/paper/2401.04722/) (2024)
- [BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks](https://mlsanity.com/paper/2305.17100/) (2023)
- [MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation](https://mlsanity.com/paper/2303.09975/) (2023)
- [BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once](https://mlsanity.com/paper/2405.12971/) (2024)
- [SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images](https://mlsanity.com/paper/2310.15161/) (2023)
- [Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge](https://mlsanity.com/paper/1902.07880/) (2019)
- [Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directions](https://mlsanity.com/paper/2404.03264/) (2024)
- [VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysis](https://mlsanity.com/paper/2402.17300/) (2024)
- [ImageCAS: A Large-Scale Dataset and Benchmark for Coronary Artery Segmentation based on Computed Tomography Angiography Images](https://mlsanity.com/paper/2211.01607/) (2022)
- [LightM-UNet: Mamba Assists in Lightweight UNet for Medical Image Segmentation](https://mlsanity.com/paper/2403.05246/) (2024)
- [Vision Foundation Models for Computed Tomography](https://mlsanity.com/paper/2501.09001/) (2025)
- [PCRLv2: A Unified Visual Information Preservation Framework for Self-supervised Pre-training in Medical Image Analysis](https://mlsanity.com/paper/2301.00772/) (2023)
- [Overcoming Data Scarcity in Biomedical Imaging with a Foundational Multi-Task Model](https://mlsanity.com/paper/2311.09847/) (2023)
- [SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2307.12591/) (2023)
- [MyoPS: A Benchmark of Myocardial Pathology Segmentation Combining Three-Sequence Cardiac Magnetic Resonance Images](https://mlsanity.com/paper/2201.03186/) (2022)
- [Deep Learning methods for automatic evaluation of delayed enhancement-MRI. The results of the EMIDEC challenge](https://mlsanity.com/paper/2108.04016/) (2021)
- [Foundation Models for Biomedical Image Segmentation: A Survey](https://mlsanity.com/paper/2401.07654/) (2024)
- [The state-of-the-art 3D anisotropic intracranial hemorrhage segmentation on non-contrast head CT: The INSTANCE challenge](https://mlsanity.com/paper/2301.03281/) (2023)
- [CMRxRecon: An open cardiac MRI dataset for the competition of accelerated image reconstruction](https://mlsanity.com/paper/2309.10836/) (2023)
- [Biomedical image analysis competitions: The state of current participation practice](https://mlsanity.com/paper/2212.08568/) (2022)
- [vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation](https://mlsanity.com/paper/2411.17386/) (2024)
- [Towards a vision foundation model for comprehensive assessment of Cardiac MRI](https://mlsanity.com/paper/2410.01665/) (2024)
- [The Extreme Cardiac MRI Analysis Challenge under Respiratory Motion (CMRxMotion)](https://mlsanity.com/paper/2210.06385/) (2022)
- [Hunting imaging biomarkers in pulmonary fibrosis: Benchmarks of the AIIB23 challenge](https://mlsanity.com/paper/2312.13752/) (2023)
- [xLSTM-UNet can be an Effective 2D & 3D Medical Image Segmentation Backbone with Vision-LSTM (ViL) better than its Mamba Counterpart](https://mlsanity.com/paper/2407.01530/) (2024)

## Built on by (most cited first, 2 of 2)

- [Towards Generalisable Foundation Models for Brain MRI](https://mlsanity.com/paper/2510.23415/) (2025)
- [BrainNext: A General-Purpose Self-Supervised Foundation Model for Brain MRI Analysis](https://mlsanity.com/paper/2607.17782/) (2026)

## Abstract (arXiv)

Whole-heart segmentation from CT and MRI scans is crucial for cardiovascular disease analysis, yet existing methods struggle with modality-specific biases and the need for extensive labeled datasets. To address these challenges, we propose a foundation model for whole-heart segmentation using a self-supervised learning (SSL) framework based on a student-teacher architecture. Our model is pretrained on a large, unlabeled dataset of CT and MRI scans, leveraging the xLSTM backbone to capture long-range spatial dependencies and complex anatomical structures in 3D medical images. By incorporating multi-modal pretraining, our approach ensures strong generalization across both CT and MRI modalities, mitigating modality-specific variations and improving segmentation accuracy in diverse clinical settings. The use of large-scale unlabeled data significantly reduces the dependency on manual annotations, enabling robust performance even with limited labeled data. We further introduce an xLSTM-UNet-based architecture for downstream whole-heart segmentation tasks, demonstrating its effectiveness on few-label CT and MRI datasets. Our results validate the robustness and adaptability of the proposed model, highlighting its potential for advancing automated whole-heart segmentation in medical imaging.