TubeMLLM: A Foundation Model for Topology Knowledge Exploration in Vessel-like Anatomy
Not on the map. Why?
Abstract (arXiv)
Modeling medical vessel-like anatomy is challenging due to its intricate topology and sensitivity to dataset shifts. Consequently, task-specific models often suffer from topological inconsistencies, including artificial disconnections and spurious merges. Motivated by the promise of multimodal large language models (MLLMs) for zero-shot generalization, we propose TubeMLLM, a unified foundation model that couples structured understanding with controllable generation for medical vessel-like anatomy. By integrating topological priors through explicit natural language prompting and aligning them with visual representations in a shared-attention architecture, TubeMLLM significantly enhances topology-aware perception. Furthermore, we construct TubeMData, a pionner multimodal benchmark comprising comprehensive topology-centric tasks, and introduce an adaptive loss weighting strategy to emphasize topology-critical regions during training. Extensive experiments on fifteen diverse datasets demonstrate our superiority. Quantitatively, TubeMLLM achieves state-of-the-art out-of-distribution performance, substantially reducing global topological discrepancies on color fundus photography (decreasing the β0 number error from 37.42 to 8.58 compared to baselines). Notably, TubeMLLM exhibits exceptional zero-shot cross-modality transferring ability on unseen X-ray angiography, achieving a Dice score of 67.50% while significantly reducing the β0 error to 1.21. TubeMLLM also maintains robustness against degradations such as blur, noise, and low resolution. Furthermore, in topology-aware understanding tasks, the model achieves 97.38% accuracy in evaluating mask topological quality, significantly outperforming standard vision-language baselines.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 57 references that are in the map. First posted on arXiv 10 Mar 2026.
Builds on
37 of 57 references.
References · 37
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Segment Anythingcited by 8,158
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2023Qwen Technical Reportcited by 3,273
- 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2025SAM 3: Segment Anything with Conceptscited by 835
- 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2023Segment Everything Everywhere All at Oncecited by 546
25 more
- 2021Domain Adaptation for Medical Image Analysis: A Surveycited by 351
- 2020clDice -- A Novel Topology-Preserving Loss Function for Tubular Structure Segmentationcited by 286
- 2024nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentationcited by 235
- 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
- 2019Topology-Preserving Deep Image Segmentationcited by 198
- 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 151
- 2019A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homologycited by 134
- 2024BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at oncecited by 107
- 2025MedSAM2: Segment Anything in 3D Medical Images and Videoscited by 88
- 2021Topology-Aware Segmentation Using Discrete Morse Theorycited by 75
- 2023Large-Vocabulary Segmentation for Medical Images with Text Promptscited by 60
- 2022Topologically faithful image segmentation via induced matching of persistence barcodescited by 48
- 2024Skeleton Recall Loss for Connectivity Conserving and Resource Efficient Segmentation of Thin Tubular Structurescited by 43
- 2018Towards a glaucoma risk index based on simulated hemodynamics from fundus imagescited by 31
- 2024Centerline Boundary Dice Loss for Vascular Segmentationcited by 27
- 2025MedSAM3: Delving into Segment Anything with Medical Conceptscited by 25
- 2026Medical SAM3: A Foundation Model for Universal Prompt-Driven Medical Image Segmentationcited by 23
- 2025VoxTell: Free-Text Promptable Universal 3D Medical Image Segmentationcited by 19
- 2024DeNVeR: Deformable Neural Vessel Representations for Unsupervised Video Vessel Segmentationcited by 13
- 2021The RETA Benchmark for Retinal Vascular Tree Analysiscited by 12
- 2024Dr-SAM: An End-to-End Framework for Vascular Segmentation, Diameter Estimation, and Anomaly Detection on Angiography Imagescited by 6
- 2025ARGenSeg: Image Segmentation with Autoregressive Image Generation Modelcited by 5
- 2023RVD: A Handheld Device-Based Fundus Video Dataset for Retinal Vessel Segmentationcited by 4
- 2026UM-Text: A Unified Multimodal Model for Image Understanding and Visual Text Editingcited by 4
- 2022Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentationcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Topo-R1: Detecting Topological Anomalies via Vision-Language Models
- 2026TopoAgent: An Agentic Framework for Automated Topology Learning in Medical Imaging
- 2025RAU: Reference-based Anatomical Understanding with Vision Language Models
- 2026TopoCL: Topological Contrastive Learning for Medical Imaging
- 2025Anatomy-R1: Enhancing Anatomy Reasoning in Multimodal Large Language Models via Anatomical Similarity Curriculum and Group Diversity Augmentation
- 2025Anatomica: Localized Control over Geometric and Topological Properties for Anatomical Diffusion Models
- 2025Anatomy-VLM: A Fine-grained Vision-Language Model for Medical Interpretation
- 2024TPIE: Topology-Preserved Image Editing With Text Instructions