# Histopathology Whole Slide Image Analysis with Heterogeneous Graph Representation Learning

arXiv: https://arxiv.org/abs/2307.04189 · HTML: https://mlsanity.com/paper/2307.04189/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 9 Jul 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Parameterized Explainer for Graph Neural Network](https://mlsanity.com/paper/2011.04573/) (2020)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [ASAP: Adaptive Structure Aware Pooling for Learning Hierarchical Graph Representations](https://mlsanity.com/paper/1911.07979/) (2019)
- [Generative Causal Explanations for Graph Neural Networks](https://mlsanity.com/paper/2104.06643/) (2021)
- [Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks](https://mlsanity.com/paper/2107.13048/) (2021)
- [PyTorch-BigGraph: A Large-scale Graph Embedding System](https://mlsanity.com/paper/1903.12287/) (2019)
- [PanNuke Dataset Extension, Insights and Baselines](https://mlsanity.com/paper/2003.10778/) (2020)
- [CXPlain: Causal Explanations for Model Interpretation under Uncertainty](https://mlsanity.com/paper/1910.12336/) (2019)
- [A graph-transformer for whole slide image classification](https://mlsanity.com/paper/2205.09671/) (2022)
- [Neural Image Compression for Gigapixel Histopathology Image Analysis](https://mlsanity.com/paper/1811.02840/) (2018)
- [A Survey on Graph-Based Deep Learning for Computational Histopathology](https://mlsanity.com/paper/2107.00272/) (2021)
- [ReMix: A General and Efficient Framework for Multiple Instance Learning based Whole Slide Image Classification](https://mlsanity.com/paper/2207.01805/) (2022)
- [RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification](https://mlsanity.com/paper/2010.06440/) (2020)
- [DiffGCN: Graph Convolutional Networks via Differential Operators and Algebraic Multigrid Pooling](https://mlsanity.com/paper/2006.04115/) (2020)
- [Bayesian Pyramids: Identifiable Multilayer Discrete Latent Structure Models for Discrete Data](https://mlsanity.com/paper/2101.10373/) (2021)
- [Understanding Overparameterization in Generative Adversarial Networks](https://mlsanity.com/paper/2104.05605/) (2021)

## Built on by (most cited first, 30 of 47)

- [Dynamic Graph Representation with Knowledge-aware Attention for Histopathology Whole Slide Image Analysis](https://mlsanity.com/paper/2403.07719/) (2024)
- [Attention-Challenging Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2311.07125/) (2023)
- [Transcriptomics-guided Slide Representation Learning in Computational Pathology](https://mlsanity.com/paper/2405.11618/) (2024)
- [ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2502.08391/) (2025)
- [Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction](https://mlsanity.com/paper/2402.19326/) (2024)
- [Knowledge-enhanced Pretraining for Vision-language Pathology Foundation Model on Cancer Diagnosis](https://mlsanity.com/paper/2412.13126/) (2024)
- [Rotation-Agnostic Image Representation Learning for Digital Pathology](https://mlsanity.com/paper/2311.08359/) (2023)
- [Knowledge-enhanced Visual-Language Pretraining for Computational Pathology](https://mlsanity.com/paper/2404.09942/) (2024)
- [Pathology-knowledge Enhanced Multi-instance Prompt Learning for Few-shot Whole Slide Image Classification](https://mlsanity.com/paper/2407.10814/) (2024)
- [Dynamic Policy-Driven Adaptive Multi-Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2403.07939/) (2024)
- [Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis](https://mlsanity.com/paper/2410.14195/) (2024)
- [Multimodal Cancer Survival Analysis via Hypergraph Learning with Cross-Modality Rebalance](https://mlsanity.com/paper/2505.11997/) (2025)
- [AEM: Attention Entropy Maximization for Multiple Instance Learning based Whole Slide Image Classification](https://mlsanity.com/paper/2406.15303/) (2024)
- [Aligning Knowledge Concepts to Whole Slide Images for Precise Histopathology Image Analysis](https://mlsanity.com/paper/2411.18101/) (2024)
- [GTP-4o: Modality-prompted Heterogeneous Graph Learning for Omni-modal Biomedical Representation](https://mlsanity.com/paper/2407.05540/) (2024)
- [cDP-MIL: Robust Multiple Instance Learning via Cascaded Dirichlet Process](https://mlsanity.com/paper/2407.11448/) (2024)
- [Long-MIL: Scaling Long Contextual Multiple Instance Learning for Histopathology Whole Slide Image Analysis](https://mlsanity.com/paper/2311.12885/) (2023)
- [PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization](https://mlsanity.com/paper/2503.06482/) (2025)
- [WISE: A Framework for Gigapixel Whole-Slide-Image Lossless Compression](https://mlsanity.com/paper/2503.18074/) (2025)
- [Cracking Instance Jigsaw Puzzles: An Alternative to Multiple Instance Learning for Whole Slide Image Analysis](https://mlsanity.com/paper/2507.08178/) (2025)
- [PTCMIL: Multiple Instance Learning via Prompt Token Clustering for Whole Slide Image Analysis](https://mlsanity.com/paper/2507.18848/) (2025)
- [Adaptive Uncertainty Estimation via High-Dimensional Testing on Latent Representations](https://mlsanity.com/paper/2310.16587/) (2023)
- [SlideGCD: Slide-based Graph Collaborative Training with Knowledge Distillation for Whole Slide Image Classification](https://mlsanity.com/paper/2407.08968/) (2024)
- [Slide-based Graph Collaborative Training for Histopathology Whole Slide Image Analysis](https://mlsanity.com/paper/2410.10260/) (2024)
- [From Histopathology Images to Cell Clouds: Learning Slide Representations with Hierarchical Cell Transformer](https://mlsanity.com/paper/2412.16715/) (2024)
- [Disentangled Multi-modal Learning of Histology and Transcriptomics for Cancer Characterization](https://mlsanity.com/paper/2508.16479/) (2025)
- [NuNext: Reframing Nucleus Detection as Next-Point Detection](https://mlsanity.com/paper/2603.07098/) (2026)
- [Predicting Genetic Mutation from Whole Slide Images via Biomedical-Linguistic Knowledge Enhanced Multi-label Classification](https://mlsanity.com/paper/2406.02990/) (2024)
- [Multi-Cohort Framework with Cohort-Aware Attention and Adversarial Mutual-Information Minimization for Whole Slide Image Classification](https://mlsanity.com/paper/2409.11119/) (2024)
- [VLEER: Vision and Language Embeddings for Explainable Whole Slide Image Representation](https://mlsanity.com/paper/2502.20850/) (2025)

## Abstract (arXiv)

Graph-based methods have been extensively applied to whole-slide histopathology image (WSI) analysis due to the advantage of modeling the spatial relationships among different entities. However, most of the existing methods focus on modeling WSIs with homogeneous graphs (e.g., with homogeneous node type). Despite their successes, these works are incapable of mining the complex structural relations between biological entities (e.g., the diverse interaction among different cell types) in the WSI. We propose a novel heterogeneous graph-based framework to leverage the inter-relationships among different types of nuclei for WSI analysis. Specifically, we formulate the WSI as a heterogeneous graph with "nucleus-type" attribute to each node and a semantic similarity attribute to each edge. We then present a new heterogeneous-graph edge attribute transformer (HEAT) to take advantage of the edge and node heterogeneity during massage aggregating. Further, we design a new pseudo-label-based semantic-consistent pooling mechanism to obtain graph-level features, which can mitigate the over-parameterization issue of conventional cluster-based pooling. Additionally, observing the limitations of existing association-based localization methods, we propose a causal-driven approach attributing the contribution of each node to improve the interpretability of our framework. Extensive experiments on three public TCGA benchmark datasets demonstrate that our framework outperforms the state-of-the-art methods with considerable margins on various tasks. Our codes are available at https://github.com/HKU-MedAI/WSI-HGNN.