# Diagnostic Text-guided Representation Learning in Hierarchical Classification for Pathological Whole Slide Image

arXiv: https://arxiv.org/abs/2411.10709 · HTML: https://mlsanity.com/paper/2411.10709/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 69 references that are in the map.
- First posted on arXiv 16 Nov 2024.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images](https://mlsanity.com/paper/2111.04740/) (2021)
- [Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks](https://mlsanity.com/paper/2107.13048/) (2021)
- [Deep Learning for Identifying Metastatic Breast Cancer](https://mlsanity.com/paper/1606.05718/) (2016)
- [Deep Learning-based Computational Pathology Predicts Origins for Cancers of Unknown Primary](https://mlsanity.com/paper/2006.13932/) (2020)
- [MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology](https://mlsanity.com/paper/2403.06800/) (2024)
- [Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology](https://mlsanity.com/paper/2405.11643/) (2024)
- [Dynamic Graph Representation with Knowledge-aware Attention for Histopathology Whole Slide Image Analysis](https://mlsanity.com/paper/2403.07719/) (2024)
- [Fine-Tuning and Training of DenseNet for Histopathology Image Representation Using TCGA Diagnostic Slides](https://mlsanity.com/paper/2101.07903/) (2021)
- [Transcriptomics-guided Slide Representation Learning in Computational Pathology](https://mlsanity.com/paper/2405.11618/) (2024)
- [Text-guided Foundation Model Adaptation for Pathological Image Classification](https://mlsanity.com/paper/2307.14901/) (2023)
- [A Comprehensive Review of Computer-aided Whole-slide Image Analysis: from Datasets to Feature Extraction, Segmentation, Classification, and Detection Approaches](https://mlsanity.com/paper/2102.10553/) (2021)
- [Streaming convolutional neural networks for end-to-end learning with multi-megapixel images](https://mlsanity.com/paper/1911.04432/) (2019)
- [The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification](https://mlsanity.com/paper/2305.17891/) (2023)
- [Diagnose Like a Pathologist: Transformer-Enabled Hierarchical Attention-Guided Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2301.08125/) (2023)
- [Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction](https://mlsanity.com/paper/2402.19326/) (2024)
- [When an Image is Worth 1,024 x 1,024 Words: A Case Study in Computational Pathology](https://mlsanity.com/paper/2312.03558/) (2023)

## Built on by (most cited first, 4 of 4)

- [Can We Simplify Slide-level Fine-tuning of Pathology Foundation Models?](https://mlsanity.com/paper/2502.20823/) (2025)
- [Multimodal Prototype Alignment for Semi-supervised Pathology Image Segmentation](https://mlsanity.com/paper/2508.19574/) (2025)
- [Unlocking adaptive digital pathology through dynamic feature learning](https://mlsanity.com/paper/2412.20430/) (2024)
- [Multimodal Distillation-Driven Ensemble Learning for Long-Tailed Histopathology Whole Slide Images Analysis](https://mlsanity.com/paper/2503.00915/) (2025)

## Abstract (arXiv)

With the development of digital imaging in medical microscopy, artificial intelligent-based analysis of pathological whole slide images (WSIs) provides a powerful tool for cancer diagnosis. Limited by the expensive cost of pixel-level annotation, current research primarily focuses on representation learning with slide-level labels, showing success in various downstream tasks. However, given the diversity of lesion types and the complex relationships between each other, these techniques still deserve further exploration in addressing advanced pathology tasks. To this end, we introduce the concept of hierarchical pathological image classification and propose a representation learning called PathTree. PathTree considers the multi-classification of diseases as a binary tree structure. Each category is represented as a professional pathological text description, which messages information with a tree-like encoder. The interactive text features are then used to guide the aggregation of hierarchical multiple representations. PathTree uses slide-text similarity to obtain probability scores and introduces two extra tree specific losses to further constrain the association between texts and slides. Through extensive experiments on three challenging hierarchical classification datasets: in-house cryosectioned lung tissue lesion identification, public prostate cancer grade assessment, and public breast cancer subtyping, our proposed PathTree is consistently competitive compared to the state-of-the-art methods and provides a new perspective on the deep learning-assisted solution for more complex WSI classification.