mlsanityData to 14 Oct 2026

HVTSurv: Hierarchical Vision Transformer for Patient-Level Survival Prediction from Whole Slide Image

arXiv 2306.17373 ↗First posted 30 Jun 2023cs.CV cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Survival prediction based on whole slide images (WSIs) is a challenging task for patient-level multiple instance learning (MIL). Due to the vast amount of data for a patient (one or multiple gigapixels WSIs) and the irregularly shaped property of WSI, it is difficult to fully explore spatial, contextual, and hierarchical interaction in the patient-level bag. Many studies adopt random sampling pre-processing strategy and WSI-level aggregation models, which inevitably lose critical prognostic information in the patient-level bag. In this work, we propose a hierarchical vision Transformer framework named HVTSurv, which can encode the local-level relative spatial information, strengthen WSI-level context-aware communication, and establish patient-level hierarchical interaction. Firstly, we design a feature pre-processing strategy, including feature rearrangement and random window masking. Then, we devise three layers to progressively obtain patient-level representation, including a local-level interaction layer adopting Manhattan distance, a WSI-level interaction layer employing spatial shuffle, and a patient-level interaction layer using attention pooling. Moreover, the design of hierarchical network helps the model become more computationally efficient. Finally, we validate HVTSurv with 3,104 patients and 3,752 WSIs across 6 cancer types from The Cancer Genome Atlas (TCGA). The average C-Index is 2.50-11.30% higher than all the prior weakly supervised methods over 6 TCGA datasets. Ablation study and attention visualization further verify the superiority of the proposed HVTSurv. Implementation is available at: https://github.com/szc19990412/HVTSurv.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 18 of its 47 references that are in the map. First posted on arXiv 30 Jun 2023.

Builds on

18 of 47 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  4. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
  5. 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 555
  6. 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 478
  7. 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 207
  8. 2020Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networkscited by 178
  9. 2019Deep neural network models for computational histopathology: A surveycited by 165
  10. 2021Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networkscited by 124
  11. 2021Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformercited by 115
  12. 2020Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Imagescited by 81
6 more
  1. 2019CAMEL: A Weakly Supervised Learning Framework for Histopathology Image Segmentationcited by 73
  2. 2020Weakly supervised multiple instance learning histopathological tumor segmentationcited by 46
  3. 2018Attention-Based Deep Neural Networks for Detection of Cancerous and Precancerous Esophagus Tissue on Histopathological Slidescited by 29
  4. 2020Divide-and-Rule: Self-Supervised Learning for Survival Analysis in Colorectal Cancercited by 17
  5. 2022Incorporating intratumoral heterogeneity into weakly-supervised deep learning models via variance poolingcited by 11
  6. 2022Multiple Instance Learning with Mixed Supervision in Gleason Gradingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 6; 2026: 3.

14 papers.

Sort:

By area:

  1. 2024SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understandingcited by 57
  2. 2024MSCPT: Few-shot Whole Slide Image Classification with Multi-scale and Context-focused Prompt Tuningcited by 16
  3. 2025MiCo: Multiple Instance Learning with Context-Aware Clustering for Whole Slide Image Analysiscited by 7
  4. 2025PS3: A Multimodal Transformer Integrating Pathology Reports with Histology Images and Biological Pathways for Cancer Survival Predictioncited by 3
  5. 2024SlideGCD: Slide-based Graph Collaborative Training with Knowledge Distillation for Whole Slide Image Classificationcited by 2
  6. 2024Slide-based Graph Collaborative Training for Histopathology Whole Slide Image Analysiscited by 2
  7. 2025Efficient Multi-Slide Visual-Language Feature Fusion for Placental Disease Classificationcited by 2
  8. 2023Point Transformer with Federated Learning for Predicting Breast Cancer HER2 Status from Hematoxylin and Eosin-Stained Whole Slide Imagescited by 1
  9. 2025Cross-Cancer Knowledge Transfer in WSI-based Prognosis Predictioncited by 1
  10. 2025Together, Then Apart: Balancing Alignment and Distinctiveness for Multimodal Survival Analysiscited by 1
  11. 2025Structural Prognostic Event Modeling for Multimodal Cancer Survival Analysiscited by 0
  12. 2026Efficient AI-Driven Multi-Section Whole Slide Image Analysis for Biochemical Recurrence Prediction in Prostate Cancercited by 0
2 more
  1. 2026Graph Mamba Survival Analysis Based on Topology-Aware orderingcited by 0
  2. 2026Morphology signal in whole slide image foundation models can automatically triage slidescited by 0