# Instance-based Vision Transformer for Subtyping of Papillary Renal Cell Carcinoma in Histopathological Image

arXiv: https://arxiv.org/abs/2106.12265 · HTML: https://mlsanity.com/paper/2106.12265/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 19 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Micro-Net: A unified model for segmentation of various objects in microscopy images](https://mlsanity.com/paper/1804.08145/) (2018)
- [Renal Cell Carcinoma Detection and Subtyping with Minimal Point-Based Annotation in Whole-Slide Images](https://mlsanity.com/paper/2008.05332/) (2020)
- [Nuclei Grading of Clear Cell Renal Cell Carcinoma in Histopathological Image by Composite High-Resolution Network](https://mlsanity.com/paper/2106.10641/) (2021)

## Built on by (most cited first, 22 of 22)

- [UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformer](https://mlsanity.com/paper/2109.04335/) (2021)
- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [A Clinical Benchmark of Public Self-Supervised Pathology Foundation Models](https://mlsanity.com/paper/2407.06508/) (2024)
- [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [PrPSeg: Universal Proposition Learning for Panoramic Renal Pathology Segmentation](https://mlsanity.com/paper/2402.19286/) (2024)
- [Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs](https://mlsanity.com/paper/2403.07743/) (2024)
- [Vision Transformers in Medical Imaging: A Review](https://mlsanity.com/paper/2211.10043/) (2022)
- [Kernel Attention Transformer (KAT) for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2206.13156/) (2022)
- [Attention De-sparsification Matters: Inducing Diversity in Digital Pathology Representation Learning](https://mlsanity.com/paper/2309.06439/) (2023)
- [Generating Progressive Images from Pathological Transitions via Diffusion Model](https://mlsanity.com/paper/2311.12316/) (2023)
- [TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction](https://mlsanity.com/paper/2301.10829/) (2023)
- [CellMix: A General Instance Relationship based Method for Data Augmentation Towards Pathology Image Classification](https://mlsanity.com/paper/2301.11513/) (2023)
- [LDCSF: Local depth convolution-based Swim framework for classifying multi-label histopathology images](https://mlsanity.com/paper/2308.10446/) (2023)
- [Deep Transfer Learning for Kidney Cancer Diagnosis](https://mlsanity.com/paper/2408.04318/) (2024)
- [CD-Net: Histopathology Representation Learning using Pyramidal Context-Detail Network](https://mlsanity.com/paper/2203.15078/) (2022)
- [Transformer-based Personalized Attention Mechanism for Medical Images with Clinical Records](https://mlsanity.com/paper/2206.03003/) (2022)
- [Shuffle Instances-based Vision Transformer for Pancreatic Cancer ROSE Image Classification](https://mlsanity.com/paper/2208.06833/) (2022)
- [IRS: Incremental Relationship-guided Segmentation for Digital Pathology](https://mlsanity.com/paper/2505.22855/) (2025)
- [PIMIP: An Open Source Platform for Pathology Information Management and Integration](https://mlsanity.com/paper/2111.05794/) (2021)
- [M2EF-NNs: Multimodal Multi-instance Evidence Fusion Neural Networks for Cancer Survival Prediction](https://mlsanity.com/paper/2408.04170/) (2024)
- [SSMamba: A Self-Supervised Hybrid State Space Model for Pathological Image Classification](https://mlsanity.com/paper/2604.15711/) (2026)

## Abstract (arXiv)

Histological subtype of papillary (p) renal cell carcinoma (RCC), type 1 vs. type 2, is an essential prognostic factor. The two subtypes of pRCC have a similar pattern, i.e., the papillary architecture, yet some subtle differences, including cellular and cell-layer level patterns. However, the cellular and cell-layer level patterns almost cannot be captured by existing CNN-based models in large-size histopathological images, which brings obstacles to directly applying these models to such a fine-grained classification task. This paper proposes a novel instance-based Vision Transformer (i-ViT) to learn robust representations of histopathological images for the pRCC subtyping task by extracting finer features from instance patches (by cropping around segmented nuclei and assigning predicted grades). The proposed i-ViT takes top-K instances as input and aggregates them for capturing both the cellular and cell-layer level patterns by a position-embedding layer, a grade-embedding layer, and a multi-head multi-layer self-attention module. To evaluate the performance of the proposed framework, experienced pathologists are invited to selected 1162 regions of interest from 171 whole slide images of type 1 and type 2 pRCC. Experimental results show that the proposed method achieves better performance than existing CNN-based models with a significant margin.