# Integrative Graph-Transformer Framework for Histopathology Whole Slide Image Representation and Classification

arXiv: https://arxiv.org/abs/2403.18134 · HTML: https://mlsanity.com/paper/2403.18134/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://mlsanity.com/paper/2006.05205/) (2020)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Rethinking Graph Transformers with Spectral Attention](https://mlsanity.com/paper/2106.03893/) (2021)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [DeeperGCN: All You Need to Train Deeper GCNs](https://mlsanity.com/paper/2006.07739/) (2020)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification](https://mlsanity.com/paper/1504.07947/) (2015)
- [BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images](https://mlsanity.com/paper/2111.04740/) (2021)
- [Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks](https://mlsanity.com/paper/2107.13048/) (2021)
- [Revisiting Multiple Instance Neural Networks](https://mlsanity.com/paper/1610.02501/) (2016)
- [A graph-transformer for whole slide image classification](https://mlsanity.com/paper/2205.09671/) (2022)
- [A Generalized Deep Learning Framework for Whole-Slide Image Segmentation and Analysis](https://mlsanity.com/paper/2001.00258/) (2020)
- [A Survey on Graph-Based Deep Learning for Computational Histopathology](https://mlsanity.com/paper/2107.00272/) (2021)
- [Diagnose Like a Pathologist: Transformer-Enabled Hierarchical Attention-Guided Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2301.08125/) (2023)
- [AMIGO: Sparse Multi-Modal Graph Transformer with Shared-Context Processing for Representation Learning of Giga-pixel Images](https://mlsanity.com/paper/2303.00865/) (2023)
- [MulGT: Multi-task Graph-Transformer with Task-aware Knowledge Injection and Domain Knowledge-driven Pooling for Whole Slide Image Analysis](https://mlsanity.com/paper/2302.10574/) (2023)
- [Multi-scale Efficient Graph-Transformer for Whole Slide Image Classification](https://mlsanity.com/paper/2305.15773/) (2023)

## Built on by (most cited first, 6 of 6)

- [A Survey of Graph Transformers: Architectures, Theories and Applications](https://mlsanity.com/paper/2502.16533/) (2025)
- [Hypergraph Mamba for Efficient Whole Slide Image Understanding](https://mlsanity.com/paper/2505.17457/) (2025)
- [EFCM: Efficient Fine-tuning on Compressed Models for deployment of large models in medical image analysis](https://mlsanity.com/paper/2409.11817/) (2024)
- [Identifying actionable driver mutations in lung cancer using an efficient Asymmetric Transformer Decoder](https://mlsanity.com/paper/2508.02431/) (2025)
- [Fourier Transform Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2510.15138/) (2025)
- [Context-aware Skin Cancer Epithelial Cell Classification with Scalable Graph Transformers](https://mlsanity.com/paper/2602.15783/) (2026)

## Abstract (arXiv)

In digital pathology, the multiple instance learning (MIL) strategy is widely used in the weakly supervised histopathology whole slide image (WSI) classification task where giga-pixel WSIs are only labeled at the slide level. However, existing attention-based MIL approaches often overlook contextual information and intrinsic spatial relationships between neighboring tissue tiles, while graph-based MIL frameworks have limited power to recognize the long-range dependencies. In this paper, we introduce the integrative graph-transformer framework that simultaneously captures the context-aware relational features and global WSI representations through a novel Graph Transformer Integration (GTI) block. Specifically, each GTI block consists of a Graph Convolutional Network (GCN) layer modeling neighboring relations at the local instance level and an efficient global attention model capturing comprehensive global information from extensive feature embeddings. Extensive experiments on three publicly available WSI datasets: TCGA-NSCLC, TCGA-RCC and BRIGHT, demonstrate the superiority of our approach over current state-of-the-art MIL methods, achieving an improvement of 1.0% to 2.6% in accuracy and 0.7%-1.6% in AUROC.