# vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation

arXiv: https://arxiv.org/abs/2411.17386 · HTML: https://mlsanity.com/paper/2411.17386/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 65 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Builds on (most cited first, 30 of 36)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [clDice -- A Novel Topology-Preserving Loss Function for Tubular Structure Segmentation](https://mlsanity.com/paper/2003.07311/) (2020)
- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation](https://mlsanity.com/paper/2303.09975/) (2023)
- [Medical SAM 2: Segment medical images as video via Segment Anything Model 2](https://mlsanity.com/paper/2408.00874/) (2024)
- [SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images](https://mlsanity.com/paper/2310.15161/) (2023)
- [3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2209.15076/) (2022)
- [Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis](https://mlsanity.com/paper/2305.18453/) (2023)
- [Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Models](https://mlsanity.com/paper/2402.05210/) (2024)
- [GenerateCT: Text-Conditional Generation of 3D Chest CT Volumes](https://mlsanity.com/paper/2305.16037/) (2023)
- [DeepVesselNet: Vessel Segmentation, Centerline Prediction, and Bifurcation Detection in 3-D Angiographic Volumes](https://mlsanity.com/paper/1803.09340/) (2018)
- [The TopCoW Challenge -- Topology-Aware Circle of Willis Segmentation for CT and MR Angiography](https://mlsanity.com/paper/2312.17670/) (2023)
- [AnyStar: Domain randomized universal star-convex 3D instance segmentation](https://mlsanity.com/paper/2307.07044/) (2023)
- [SAM-OCTA: A Fine-Tuning Strategy for Applying Foundation Model to OCTA Image Segmentation Tasks](https://mlsanity.com/paper/2309.11758/) (2023)
- [Deep Generative Models for 3D Medical Image Synthesis](https://mlsanity.com/paper/2410.17664/) (2024)
- [Deep convolutional neural networks for segmenting 3D in vivo multiphoton images of vasculature in Alzheimer disease mouse models](https://mlsanity.com/paper/1801.00880/) (2018)
- [Automatic Segmentation, Feature Extraction and Comparison of Healthy and Stroke Cerebral Vasculature](https://mlsanity.com/paper/2002.11208/) (2020)
- [Dr-SAM: An End-to-End Framework for Vascular Segmentation, Diameter Estimation, and Anomaly Detection on Angiography Images](https://mlsanity.com/paper/2404.17029/) (2024)
- [Simulation-Based Segmentation of Blood Vessels in Cerebral 3D OCTA Images](https://mlsanity.com/paper/2403.07116/) (2024)

## Built on by (most cited first, 18 of 18)

- [Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation](https://mlsanity.com/paper/2512.18176/) (2025)
- [Foundation Model for Whole-Heart Segmentation: Leveraging Student-Teacher Learning in Multi-Modal Medical Imaging](https://mlsanity.com/paper/2503.19005/) (2025)
- [VesselSDF: Distance Field Priors for Vascular Network Reconstruction](https://mlsanity.com/paper/2506.16556/) (2025)
- [Semantically Consistent Discrete Diffusion for 3D Biological Graph Modeling](https://mlsanity.com/paper/2507.04856/) (2025)
- [RefTr: Recurrent Refinement of Confluent Trajectories for 3D Vascular Tree Centerlines](https://mlsanity.com/paper/2511.20823/) (2025)
- [VesselTok: Tokenizing Vessel-like 3D Biomedical Graph Representations for Reconstruction and Generation](https://mlsanity.com/paper/2603.18797/) (2026)
- [COMMA: Coordinate-aware Modulated Mamba Network for 3D Dispersed Vessel Segmentation](https://mlsanity.com/paper/2503.02332/) (2025)
- [Synthetic Data for Blood Vessel Network Extraction](https://mlsanity.com/paper/2504.11858/) (2025)
- [Robust automatic brain vessel segmentation in 3D CTA scans using dynamic 4D-CTA data](https://mlsanity.com/paper/2602.00391/) (2026)
- [FalconBC: Flow matching for Amortized inference of Latent-CONditioned physiologic Boundary Conditions](https://mlsanity.com/paper/2603.19331/) (2026)
- [Sparse Representation Learning for Vessels](https://mlsanity.com/paper/2605.01382/) (2026)
- [Bridging 3D Deep Learning and Curation for Analysis and High-Quality Segmentation in Practice](https://mlsanity.com/paper/2511.22236/) (2025)
- [SemiSAM-O1: Pushing the Boundary of Annotation-Efficient Medical Image Segmentation with Generalist Knowledge Fusion](https://mlsanity.com/paper/2604.24109/) (2026)
- [Vesselpose: Vessel Graph Reconstruction from Learned Voxel-wise Direction Vectors in 3D Vascular Images](https://mlsanity.com/paper/2605.00538/) (2026)
- [VesselSim: learning 3D blood vessel segmentation without expert annotations](https://mlsanity.com/paper/2605.26277/) (2026)
- [vesselFM-CT: Segmenting All Blood Vessels in CT Images for System-Level Cardiovascular Analysis](https://mlsanity.com/paper/2606.09400/) (2026)
- [Flow Matching Meets 3D Curvilinear Structure Segmentation in Medical Imaging](https://mlsanity.com/paper/2608.19965/) (2026)
- [Evaluation Principles for MRI-MRA Registration in Trigeminal Neuralgia: An ROI-Centered Neurovascular Benchmark](https://mlsanity.com/paper/2609.08805/) (2026)

## Abstract (arXiv)

Segmenting 3D blood vessels is a critical yet challenging task in medical image analysis. This is due to significant imaging modality-specific variations in artifacts, vascular patterns and scales, signal-to-noise ratios, and background tissues. These variations, along with domain gaps arising from varying imaging protocols, limit the generalization of existing supervised learning-based methods, requiring tedious voxel-level annotations for each dataset separately. While foundation models promise to alleviate this limitation, they typically fail to generalize to the task of blood vessel segmentation, posing a unique, complex problem. In this work, we present vesselFM, a foundation model designed specifically for the broad task of 3D blood vessel segmentation. Unlike previous models, vesselFM can effortlessly generalize to unseen domains. To achieve zero-shot generalization, we train vesselFM on three heterogeneous data sources: a large, curated annotated dataset, data generated by a domain randomization scheme, and data sampled from a flow matching-based generative model. Extensive evaluations show that vesselFM outperforms state-of-the-art medical image segmentation foundation models across four (pre-)clinically relevant imaging modalities in zero-, one-, and few-shot scenarios, therefore providing a universal solution for 3D blood vessel segmentation.