# Semi-supervised ViT knowledge distillation network with style transfer normalization for colorectal liver metastases survival prediction

arXiv: https://arxiv.org/abs/2311.10305 · HTML: https://mlsanity.com/paper/2311.10305/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 61 references that are in the map.
- First posted on arXiv 17 Nov 2023.

## Builds on (most cited first, 28 of 28)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network](https://mlsanity.com/paper/1606.00931/) (2016)
- [Billion-scale semi-supervised learning for image classification](https://mlsanity.com/paper/1905.00546/) (2019)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Global Context Vision Transformers](https://mlsanity.com/paper/2206.09959/) (2022)
- [Knowledge Distillation via the Target-aware Transformer](https://mlsanity.com/paper/2205.10793/) (2022)
- [Deep Learning for Whole Slide Image Analysis: An Overview](https://mlsanity.com/paper/1910.11097/) (2019)
- [Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology](https://mlsanity.com/paper/2203.00585/) (2022)
- [Deep learning-based survival prediction for multiple cancer types using histopathology images](https://mlsanity.com/paper/1912.07354/) (2019)
- [A Generalized Deep Learning Framework for Whole-Slide Image Segmentation and Analysis](https://mlsanity.com/paper/2001.00258/) (2020)
- [A Comprehensive Review of Computer-aided Whole-slide Image Analysis: from Datasets to Feature Extraction, Segmentation, Classification, and Detection Approaches](https://mlsanity.com/paper/2102.10553/) (2021)
- [Going to Extremes: Weakly Supervised Medical Image Segmentation](https://mlsanity.com/paper/2009.11988/) (2020)
- [ViTKD: Practical Guidelines for ViT feature knowledge distillation](https://mlsanity.com/paper/2209.02432/) (2022)
- [Interpretable Survival Prediction for Colorectal Cancer using Deep Learning](https://mlsanity.com/paper/2011.08965/) (2020)
- [GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathological Image Detection](https://mlsanity.com/paper/2104.14528/) (2021)
- [The importance of stain normalization in colorectal tissue classification with convolutional networks](https://mlsanity.com/paper/1702.05931/) (2017)
- [MIML-FCN+: Multi-instance Multi-label Learning via Fully Convolutional Networks with Privileged Information](https://mlsanity.com/paper/1702.08681/) (2017)
- [Teacher-Student chain for efficient semi-supervised histology image classification](https://mlsanity.com/paper/2003.08797/) (2020)
- [Boosting Contrastive Learning with Relation Knowledge Distillation](https://mlsanity.com/paper/2112.04174/) (2021)
- [Stain Style Transfer of Histopathology Images Via Structure-Preserved Generative Learning](https://mlsanity.com/paper/2007.12578/) (2020)
- [When will the mist clear? On the Interpretability of Machine Learning for Medical Applications: a survey](https://mlsanity.com/paper/2010.00353/) (2020)

## Abstract (arXiv)

Colorectal liver metastases (CLM) significantly impact colon cancer patients, influencing survival based on systemic chemotherapy response. Traditional methods like tumor grading scores (e.g., tumor regression grade - TRG) for prognosis suffer from subjectivity, time constraints, and expertise demands. Current machine learning approaches often focus on radiological data, yet the relevance of histological images for survival predictions, capturing intricate tumor microenvironment characteristics, is gaining recognition. To address these limitations, we propose an end-to-end approach for automated prognosis prediction using histology slides stained with H&E and HPS. We first employ a Generative Adversarial Network (GAN) for slide normalization to reduce staining variations and improve the overall quality of the images that are used as input to our prediction pipeline. We propose a semi-supervised model to perform tissue classification from sparse annotations, producing feature maps. We use an attention-based approach that weighs the importance of different slide regions in producing the final classification results. We exploit the extracted features for the metastatic nodules and surrounding tissue to train a prognosis model. In parallel, we train a vision Transformer (ViT) in a knowledge distillation framework to replicate and enhance the performance of the prognosis prediction. In our evaluation on a clinical dataset of 258 patients, our approach demonstrates superior performance with c-indexes of 0.804 (0.014) for OS and 0.733 (0.014) for TTR. Achieving 86.9% to 90.3% accuracy in predicting TRG dichotomization and 78.5% to 82.1% accuracy for the 3-class TRG classification task, our approach outperforms comparative methods. Our proposed pipeline can provide automated prognosis for pathologists and oncologists, and can greatly promote precision medicine progress in managing CLM patients.