# Multi-Scale Relational Graph Convolutional Network for Multiple Instance Learning in Histopathology Images

arXiv: https://arxiv.org/abs/2212.08781 · HTML: https://mlsanity.com/paper/2212.08781/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 65 references that are in the map.
- First posted on arXiv 17 Dec 2022.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [Self-Attention Graph Pooling](https://mlsanity.com/paper/1904.08082/) (2019)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology](https://mlsanity.com/paper/1902.06543/) (2019)
- [Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification](https://mlsanity.com/paper/1504.07947/) (2015)
- [Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks](https://mlsanity.com/paper/2107.13048/) (2021)
- [A graph-transformer for whole slide image classification](https://mlsanity.com/paper/2205.09671/) (2022)
- [Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images](https://mlsanity.com/paper/2001.01599/) (2020)
- [Neural Image Compression for Gigapixel Histopathology Image Analysis](https://mlsanity.com/paper/1811.02840/) (2018)
- [Hierarchical Graph Representations in Digital Pathology](https://mlsanity.com/paper/2102.11057/) (2021)
- [Query Adaptive Few-Shot Object Detection with Heterogeneous Graph Convolutional Networks](https://mlsanity.com/paper/2112.09791/) (2021)
- [Contextual Heterogeneous Graph Network for Human-Object Interaction Detection](https://mlsanity.com/paper/2010.10001/) (2020)
- [Differentiable Zooming for Multiple Instance Learning on Whole-Slide Images](https://mlsanity.com/paper/2204.12454/) (2022)
- [A Survey on Graph-Based Deep Learning for Computational Histopathology](https://mlsanity.com/paper/2107.00272/) (2021)
- [Representation Learning of Histopathology Images using Graph Neural Networks](https://mlsanity.com/paper/2004.07399/) (2020)

## Built on by (most cited first, 6 of 6)

- [Graph Neural Networks in Histopathology: Emerging Trends and Future Directions](https://mlsanity.com/paper/2406.12808/) (2024)
- [Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathology](https://mlsanity.com/paper/2603.08328/) (2026)
- [Integrating multiscale topology in digital pathology with pyramidal graph convolutional networks](https://mlsanity.com/paper/2403.15068/) (2024)
- [Multi-Resolution Histopathology Patch Graphs for Ovarian Cancer Subtyping](https://mlsanity.com/paper/2407.18105/) (2024)
- [Multi-View Stenosis Classification Leveraging Transformer-Based Multiple-Instance Learning Using Real-World Clinical Data](https://mlsanity.com/paper/2602.02067/) (2026)
- [Can We Go Beyond Visual Features? Neural Tissue Relation Modeling for Relational Graph Analysis in Non-Melanoma Skin Histology](https://mlsanity.com/paper/2512.06949/) (2025)

## Abstract (arXiv)

Graph convolutional neural networks have shown significant potential in natural and histopathology images. However, their use has only been studied in a single magnification or multi-magnification with late fusion. In order to leverage the multi-magnification information and early fusion with graph convolutional networks, we handle different embedding spaces at each magnification by introducing the Multi-Scale Relational Graph Convolutional Network (MS-RGCN) as a multiple instance learning method. We model histopathology image patches and their relation with neighboring patches and patches at other scales (i.e., magnifications) as a graph. To pass the information between different magnification embedding spaces, we define separate message-passing neural networks based on the node and edge type. We experiment on prostate cancer histopathology images to predict the grade groups based on the extracted features from patches. We also compare our MS-RGCN with multiple state-of-the-art methods with evaluations on several source and held-out datasets. Our method outperforms the state-of-the-art on all of the datasets and image types consisting of tissue microarrays, whole-mount slide regions, and whole-slide images. Through an ablation study, we test and show the value of the pertinent design features of the MS-RGCN.