Zero-shot segmentation of skin tumors in whole-slide images with vision-language foundation models
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Accurate annotation of cutaneous neoplasm biopsies represents a major challenge due to their wide morphological variability, overlapping histological patterns, and the subtle distinctions between benign and malignant lesions. Vision-language foundation models (VLMs), pre-trained on paired image-text corpora, learn joint representations that bridge visual features and diagnostic terminology, enabling zero-shot localization and classification of tissue regions without pixel-level labels. However, most existing VLM applications in histopathology remain limited to slide-level tasks or rely on coarse interactive prompts, and they struggle to produce fine-grained segmentations across gigapixel whole-slide images (WSIs). In this work, we introduce a zero-shot visual-language segmentation pipeline for whole-slide images (ZEUS), a fully automated, zero-shot segmentation framework that leverages class-specific textual prompt ensembles and frozen VLM encoders to generate high-resolution tumor masks in WSIs. By partitioning each WSI into overlapping patches, extracting visual embeddings, and computing cosine similarities against text prompts, we generate a final segmentation mask. We demonstrate competitive performance on two in-house datasets, primary spindle cell neoplasms and cutaneous metastases, highlighting the influence of prompt design, domain shifts, and institutional variability in VLMs for histopathology. ZEUS markedly reduces annotation burden while offering scalable, explainable tumor delineation for downstream diagnostic workflows.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 16 references that are in the map. First posted on arXiv 24 Nov 2025.
Builds on
9 of 16 references.
Closest ancestors · 2
Same direction, newest first
- 2025Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design30 Apr
- 2024Knowledge-enhanced Pretraining for Vision-language Pathology Foundation Model on Cancer Diagnosis17 Dec
Other references · 7
- 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
- 2025Current Pathology Foundation Models are unrobust to Medical Center Differencescited by 39
- 2025Foundation Models in Computational Pathology: A Review of Challenges, Opportunities, and Impactcited by 22
- 2023Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentationcited by 17
- 2024WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide imagescited by 7
- 2024PFPs: Prompt-guided Flexible Pathological Segmentation for Diverse Potential Outcomes Using Large Vision and Language Modelscited by 4
- 2025PathSegDiff: Pathology Segmentation using Diffusion model representationscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Effortless Vision-Language Model Specialization in Histopathology without Annotation
- 2025Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathology
- 2024Boosting Vision-Language Models for Histopathology Classification: Predict all at once
- 2025Leveraging Vision-Language Embeddings for Zero-Shot Learning in Histopathology Images
- 2026Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images
- 2024MI-VisionShot: Few-shot adaptation of vision-language models for slide-level classification of histopathological images
- 2025Democratising Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modelling
- 2026Vision-Language Models as Zero-Annotation Oracles in Histopathology