Multimodal Feature Prototype Learning for Interpretable and Discriminative Cancer Survival Prediction
Not on the map. Why?
Abstract (arXiv)
Survival analysis plays a vital role in making clinical decisions. However, the models currently in use are often difficult to interpret, which reduces their usefulness in clinical settings. Prototype learning presents a potential solution, yet traditional methods focus on local similarities and static matching, neglecting the broader tumor context and lacking strong semantic alignment with genomic data. To overcome these issues, we introduce an innovative prototype-based multimodal framework, FeatProto, aimed at enhancing cancer survival prediction by addressing significant limitations in current prototype learning methodologies within pathology. Our framework establishes a unified feature prototype space that integrates both global and local features of whole slide images (WSI) with genomic profiles. This integration facilitates traceable and interpretable decision-making processes. Our approach includes three main innovations: (1) A robust phenotype representation that merges critical patches with global context, harmonized with genomic data to minimize local bias. (2) An Exponential Prototype Update Strategy (EMA ProtoUp) that sustains stable cross-modal associations and employs a wandering mechanism to adapt prototypes flexibly to tumor heterogeneity. (3) A hierarchical prototype matching scheme designed to capture global centrality, local typicality, and cohort-level trends, thereby refining prototype inference. Comprehensive evaluations on four publicly available cancer datasets indicate that our method surpasses current leading unimodal and multimodal survival prediction techniques in both accuracy and interpretability, providing a new perspective on prototype learning for critical medical applications. Our source code is available at https://github.com/JSLiam94/FeatProto.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 64 references that are in the map. First posted on arXiv 7 Oct 2025.
Builds on
36 of 64 references.
References · 36
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2017Prototypical Networks for Few-shot Learningcited by 3,001
- 2018Attention-based Deep Multiple Instance Learningcited by 1,077
- 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
- 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
- 2017Self-Normalizing Neural Networkscited by 591
- 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 549
- 2020Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networkscited by 178
- 2023Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Predictioncited by 92
- 2023Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Predictioncited by 86
- 2023Deep Learning for Survival Analysis: A Reviewcited by 75
24 more
- 2023Cross-Modal Translation and Alignment for Survival Analysiscited by 70
- 2024Multimodal Prototyping for cancer survival predictioncited by 47
- 2024Prototypical Information Bottlenecking and Disentangling for Multimodal Cancer Survival Predictioncited by 41
- 2024Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interactioncited by 27
- 2024Interpretable Image Classification with Adaptive Prototype-based Vision Transformerscited by 25
- 2024MoME: Mixture of Multimodal Experts for Cancer Survival Predictioncited by 23
- 2024Accurate Leukocyte Detection Based on Deformable-DETR and Multi-Level Feature Fusion for Aiding Diagnosis of Blood Diseasescited by 15
- 2024Cohort-Individual Cooperative Learning for Multimodal Cancer Survival Analysiscited by 13
- 2025Robust Multimodal Survival Prediction with the Latent Differentiation Conditional Variational AutoEncodercited by 13
- 2024Dynamic Policy-Driven Adaptive Multi-Instance Learning for Whole Slide Image Classificationcited by 12
- 2023Interpreting and Correcting Medical Image Classification with PIP-Netcited by 11
- 2023Mixture of Gaussian-distributed Prototypes with Generative Modelling for Interpretable and Trustworthy Image Recognitioncited by 11
- 2024MCPNet: An Interpretable Classifier via Multi-Level Concept Prototypescited by 10
- 2024SCKansformer: Fine-Grained Classification of Bone Marrow Cells via Kansformer Backbone and Hierarchical Attention Mechanismscited by 10
- 2024Queryable Prototype Multiple Instance Learning with Vision-Language Models for Incremental Whole Slide Image Classificationcited by 10
- 2022The Concordance Index decomposition: A measure for a deeper understanding of survival prediction modelscited by 8
- 2023Deep Copula-Based Survival Analysis for Dependent Censoring with Identifiability Guaranteescited by 8
- 2024SCMIL: Sparse Context-aware Multiple Instance Learning for Predicting Cancer Survival Probability Distribution in Whole Slide Imagescited by 8
- 2023ProtoArgNet: Interpretable Image Classification with Super-Prototypes and Argumentation [Technical Report]cited by 7
- 2024This actually looks like that: Proto-BagNets for local and global interpretability-by-designcited by 7
- 2023Interpretable Medical Image Classification using Prototype Learning and Privileged Informationcited by 6
- 2025Geodesic Prototype Matching via Diffusion Maps for Interpretable Fine-Grained Recognitioncited by 6
- 2025MMLNB: Multi-Modal Learning for Neuroblastoma Subtyping Classification Assisted with Textual Description Generationcited by 3
- 2023Single-cell Multi-view Clustering via Community Detection with Unknown Number of Clusterscited by 2
Built on by
Not cited yet.