# Prototype-Based Image Prompting for Weakly Supervised Histopathological Image Segmentation

arXiv: https://arxiv.org/abs/2503.12068 · HTML: https://mlsanity.com/paper/2503.12068/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 60 references that are in the map.
- First posted on arXiv 15 Mar 2025.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment](https://mlsanity.com/paper/1908.06391/) (2019)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [OpenPrompt: An Open-source Framework for Prompt-learning](https://mlsanity.com/paper/2111.01998/) (2021)
- [Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2103.08896/) (2021)
- [Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2105.08965/) (2021)
- [Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.00962/) (2022)
- [Cross Language Image Matching for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02668/) (2022)
- [Constrained-CNN losses for weakly supervised segmentation](https://mlsanity.com/paper/1805.04628/) (2018)
- [Weakly Supervised Semantic Segmentation using Out-of-Distribution Data](https://mlsanity.com/paper/2203.03860/) (2022)
- [Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images](https://mlsanity.com/paper/2007.05448/) (2020)
- [Universal-Prototype Enhancing for Few-Shot Object Detection](https://mlsanity.com/paper/2103.01077/) (2021)
- [Multi-Layer Pseudo-Supervision for Histopathology Tissue Semantic Segmentation using Patch-level Classification Labels](https://mlsanity.com/paper/2110.08048/) (2021)
- [Bounding boxes for weakly supervised segmentation: Global constraints get close to full supervision](https://mlsanity.com/paper/2004.06816/) (2020)
- [Foundation Model Assisted Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2312.03585/) (2023)
- [Transformer based multiple instance learning for weakly supervised histopathology image segmentation](https://mlsanity.com/paper/2205.08878/) (2022)
- [SAM-Path: A Segment Anything Model for Semantic Segmentation in Digital Pathology](https://mlsanity.com/paper/2307.09570/) (2023)
- [Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generation](https://mlsanity.com/paper/2403.05239/) (2024)

## Built on by (most cited first, 17 of 17)

- [Multimodal Cancer Survival Analysis via Hypergraph Learning with Cross-Modality Rebalance](https://mlsanity.com/paper/2505.11997/) (2025)
- [Hypergraph Tversky-Aware Domain Incremental Learning for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2505.16809/) (2025)
- [Spatially Gene Expression Prediction using Dual-Scale Contrastive Learning](https://mlsanity.com/paper/2506.23827/) (2025)
- [Memory-Augmented Incomplete Multimodal Survival Prediction via Cross-Slide and Gene-Attentive Hypergraph Learning](https://mlsanity.com/paper/2506.19324/) (2025)
- [LPD: Learnable Prototypes with Diversity Regularization for Weakly Supervised Histopathology Segmentation](https://mlsanity.com/paper/2512.05922/) (2025)
- [Flexible Concept Bottleneck Model](https://mlsanity.com/paper/2511.06678/) (2025)
- [Emerging Trends in Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation with Image-Level Supervision](https://mlsanity.com/paper/2507.21587/) (2025)
- [Multi-Sequence Parotid Gland Lesion Segmentation via Expert Text-Guided Segment Anything Model](https://mlsanity.com/paper/2508.09645/) (2025)
- [Cross-Stain Contrastive Learning for Paired Immunohistochemistry and Histopathology Slide Representation Learning](https://mlsanity.com/paper/2512.03577/) (2025)
- [Harnessing Hypergraphs in Geometric Deep Learning for 3D RNA Inverse Folding](https://mlsanity.com/paper/2512.03592/) (2025)
- [DualProtoSeg: Simple and Efficient Design with Text- and Image-Guided Prototype Learning for Weakly Supervised Histopathology Image Segmentation](https://mlsanity.com/paper/2512.10314/) (2025)
- [ConStruct: Structural Distillation of Foundation Models for Prototype-Based Weakly Supervised Histopathology Segmentation](https://mlsanity.com/paper/2512.10316/) (2025)
- [VitaminP: cross-modal learning enables whole-cell segmentation from routine histology](https://mlsanity.com/paper/2604.23799/) (2026)
- [Single-Stage Hierarchical Rectification for Weakly Supervised Histopathology Segmentation](https://mlsanity.com/paper/2606.20250/) (2026)
- [C2RM-Seg: Causal Counterfactual Reasoning with Structural-Semantic Priors for Weakly Supervised Histopathological Tissue Segmentation](https://mlsanity.com/paper/2606.25508/) (2026)
- [ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation](https://mlsanity.com/paper/2608.11765/) (2026)
- [VL-AcneSeg: A Vision-Language Framework for Region-Aware Acne Lesion Segmentation](https://mlsanity.com/paper/2609.34472/) (2026)

## Abstract (arXiv)

Weakly supervised image segmentation with image-level labels has drawn attention due to the high cost of pixel-level annotations. Traditional methods using Class Activation Maps (CAMs) often highlight only the most discriminative regions, leading to incomplete masks. Recent approaches that introduce textual information struggle with histopathological images due to inter-class homogeneity and intra-class heterogeneity. In this paper, we propose a prototype-based image prompting framework for histopathological image segmentation. It constructs an image bank from the training set using clustering, extracting multiple prototype features per class to capture intra-class heterogeneity. By designing a matching loss between input features and class-specific prototypes using contrastive learning, our method addresses inter-class homogeneity and guides the model to generate more accurate CAMs. Experiments on four datasets (LUAD-HistoSeg, BCSS-WSSS, GCSS, and BCSS) show that our method outperforms existing weakly supervised segmentation approaches, setting new benchmarks in histopathological image segmentation.