mlsanityData to 7 Oct 2026

TubeMLLM: A Foundation Model for Topology Knowledge Exploration in Vessel-like Anatomy

arXiv 2603.09217 ↗First posted 10 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Modeling medical vessel-like anatomy is challenging due to its intricate topology and sensitivity to dataset shifts. Consequently, task-specific models often suffer from topological inconsistencies, including artificial disconnections and spurious merges. Motivated by the promise of multimodal large language models (MLLMs) for zero-shot generalization, we propose TubeMLLM, a unified foundation model that couples structured understanding with controllable generation for medical vessel-like anatomy. By integrating topological priors through explicit natural language prompting and aligning them with visual representations in a shared-attention architecture, TubeMLLM significantly enhances topology-aware perception. Furthermore, we construct TubeMData, a pionner multimodal benchmark comprising comprehensive topology-centric tasks, and introduce an adaptive loss weighting strategy to emphasize topology-critical regions during training. Extensive experiments on fifteen diverse datasets demonstrate our superiority. Quantitatively, TubeMLLM achieves state-of-the-art out-of-distribution performance, substantially reducing global topological discrepancies on color fundus photography (decreasing the β0 number error from 37.42 to 8.58 compared to baselines). Notably, TubeMLLM exhibits exceptional zero-shot cross-modality transferring ability on unseen X-ray angiography, achieving a Dice score of 67.50% while significantly reducing the β0 error to 1.21. TubeMLLM also maintains robustness against degradations such as blur, noise, and low resolution. Furthermore, in topology-aware understanding tasks, the model achieves 97.38% accuracy in evaluating mask topological quality, significantly outperforming standard vision-language baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 57 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

37 of 57 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Segment Anythingcited by 8,158
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2023Qwen Technical Reportcited by 3,273
  6. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  7. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  8. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  9. 2025SAM 3: Segment Anything with Conceptscited by 835
  10. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  11. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  12. 2023Segment Everything Everywhere All at Oncecited by 546
25 more
  1. 2021Domain Adaptation for Medical Image Analysis: A Surveycited by 351
  2. 2020clDice -- A Novel Topology-Preserving Loss Function for Tubular Structure Segmentationcited by 286
  3. 2024nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentationcited by 235
  4. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
  5. 2019Topology-Preserving Deep Image Segmentationcited by 198
  6. 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 151
  7. 2019A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homologycited by 134
  8. 2024BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at oncecited by 107
  9. 2025MedSAM2: Segment Anything in 3D Medical Images and Videoscited by 88
  10. 2021Topology-Aware Segmentation Using Discrete Morse Theorycited by 75
  11. 2023Large-Vocabulary Segmentation for Medical Images with Text Promptscited by 60
  12. 2022Topologically faithful image segmentation via induced matching of persistence barcodescited by 48
  13. 2024Skeleton Recall Loss for Connectivity Conserving and Resource Efficient Segmentation of Thin Tubular Structurescited by 43
  14. 2018Towards a glaucoma risk index based on simulated hemodynamics from fundus imagescited by 31
  15. 2024Centerline Boundary Dice Loss for Vascular Segmentationcited by 27
  16. 2025MedSAM3: Delving into Segment Anything with Medical Conceptscited by 25
  17. 2026Medical SAM3: A Foundation Model for Universal Prompt-Driven Medical Image Segmentationcited by 23
  18. 2025VoxTell: Free-Text Promptable Universal 3D Medical Image Segmentationcited by 19
  19. 2024DeNVeR: Deformable Neural Vessel Representations for Unsupervised Video Vessel Segmentationcited by 13
  20. 2021The RETA Benchmark for Retinal Vascular Tree Analysiscited by 12
  21. 2024Dr-SAM: An End-to-End Framework for Vascular Segmentation, Diameter Estimation, and Anomaly Detection on Angiography Imagescited by 6
  22. 2025ARGenSeg: Image Segmentation with Autoregressive Image Generation Modelcited by 5
  23. 2023RVD: A Handheld Device-Based Fundus Video Dataset for Retinal Vessel Segmentationcited by 4
  24. 2026UM-Text: A Unified Multimodal Model for Image Understanding and Visual Text Editingcited by 4
  25. 2022Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Topo-R1: Detecting Topological Anomalies via Vision-Language Models
  2. 2026TopoAgent: An Agentic Framework for Automated Topology Learning in Medical Imaging
  3. 2025RAU: Reference-based Anatomical Understanding with Vision Language Models
  4. 2026TopoCL: Topological Contrastive Learning for Medical Imaging
  5. 2025Anatomy-R1: Enhancing Anatomy Reasoning in Multimodal Large Language Models via Anatomical Similarity Curriculum and Group Diversity Augmentation
  6. 2025Anatomica: Localized Control over Geometric and Topological Properties for Anatomical Diffusion Models
  7. 2025Anatomy-VLM: A Fine-grained Vision-Language Model for Medical Interpretation
  8. 2024TPIE: Topology-Preserved Image Editing With Text Instructions