# Interpretable Survival Prediction for Colorectal Cancer using Deep Learning

arXiv: https://arxiv.org/abs/2011.08965 · HTML: https://mlsanity.com/paper/2011.08965/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2020.

## Built on by (most cited first, 30 of 30)

- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks](https://mlsanity.com/paper/2107.13048/) (2021)
- [Artificial Intelligence for Digital and Computational Pathology](https://mlsanity.com/paper/2401.06148/) (2023)
- [A General-Purpose Self-Supervised Model for Computational Pathology](https://mlsanity.com/paper/2308.15474/) (2023)
- [Robust and Efficient Medical Imaging with Self-Supervision](https://mlsanity.com/paper/2205.09723/) (2022)
- [Self-Supervised Representation Learning using Visual Field Expansion on Digital Pathology](https://mlsanity.com/paper/2109.03299/) (2021)
- [AdvMIL: Adversarial Multiple Instance Learning for the Survival Analysis on Whole-Slide Images](https://mlsanity.com/paper/2212.06515/) (2022)
- [MHAttnSurv: Multi-Head Attention for Survival Prediction Using Whole-Slide Pathology Images](https://mlsanity.com/paper/2110.11558/) (2021)
- [Incorporating intratumoral heterogeneity into weakly-supervised deep learning models via variance pooling](https://mlsanity.com/paper/2206.08885/) (2022)
- [Domain-specific optimization and diverse evaluation of self-supervised models for histopathology](https://mlsanity.com/paper/2310.13259/) (2023)
- [Diffusion-based generation of Histopathological Whole Slide Images at a Gigapixel scale](https://mlsanity.com/paper/2311.08199/) (2023)
- [Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy](https://mlsanity.com/paper/2306.07999/) (2023)
- [A Framework for Interpretability in Machine Learning for Medical Imaging](https://mlsanity.com/paper/2310.01685/) (2023)
- [PathOrchestra: A Comprehensive Foundation Model for Computational Pathology with Over 100 Diverse Clinical-Grade Tasks](https://mlsanity.com/paper/2503.24345/) (2025)
- [MGCT: Mutual-Guided Cross-Modality Transformer for Survival Outcome Prediction using Integrative Histopathology-Genomic Features](https://mlsanity.com/paper/2311.11659/) (2023)
- [Hierarchical Optimal Transport for Comparing Histopathology Datasets](https://mlsanity.com/paper/2204.08324/) (2022)
- [Interpretable Prediction of Lung Squamous Cell Carcinoma Recurrence With Self-supervised Learning](https://mlsanity.com/paper/2203.12204/) (2022)
- [A Retrospective Analysis using Deep-Learning Models for Prediction of Survival Outcome and Benefit of Adjuvant Chemotherapy in Stage II/III Colorectal Cancer](https://mlsanity.com/paper/2111.03532/) (2021)
- [Trucks Don't Mean Trump: Diagnosing Human Error in Image Analysis](https://mlsanity.com/paper/2205.07333/) (2022)
- [Colorectal cancer survival prediction using deep distribution based multiple-instance learning](https://mlsanity.com/paper/2204.11294/) (2022)
- [Deep Neural Networks integrating genomics and histopathological images for predicting stages and survival time-to-event in colon cancer](https://mlsanity.com/paper/2212.06834/) (2022)
- [Semi-supervised variational autoencoder for cell feature extraction in multiplexed immunofluorescence images](https://mlsanity.com/paper/2406.15727/) (2024)
- [HistoGym: A Reinforcement Learning Environment for Histopathological Image Analysis](https://mlsanity.com/paper/2408.08847/) (2024)
- [Prototype-driven fusion of pathology and spatial transcriptomics for interpretable survival prediction](https://mlsanity.com/paper/2602.12441/) (2026)
- [Automated risk classification of colon biopsies based on semantic segmentation of histopathology images](https://mlsanity.com/paper/2109.07892/) (2021)
- [The State of Applying Artificial Intelligence to Tissue Imaging for Cancer Research and Early Detection](https://mlsanity.com/paper/2306.16989/) (2023)
- [Semi-supervised ViT knowledge distillation network with style transfer normalization for colorectal liver metastases survival prediction](https://mlsanity.com/paper/2311.10305/) (2023)
- [NearbyPatchCL: Leveraging Nearby Patches for Self-Supervised Patch-Level Multi-Class Classification in Whole-Slide Images](https://mlsanity.com/paper/2312.07489/) (2023)
- [From Explainable to Explained AI: Ideas for Falsifying and Quantifying Explanations](https://mlsanity.com/paper/2508.09205/) (2025)
- [GAS-MIL: Group-Aggregative Selection Multi-Instance Learning for Ensemble of Foundation Models in Digital Pathology Image Analysis](https://mlsanity.com/paper/2510.03555/) (2025)

## Abstract (arXiv)

Deriving interpretable prognostic features from deep-learning-based prognostic histopathology models remains a challenge. In this study, we developed a deep learning system (DLS) for predicting disease specific survival for stage II and III colorectal cancer using 3,652 cases (27,300 slides). When evaluated on two validation datasets containing 1,239 cases (9,340 slides) and 738 cases (7,140 slides) respectively, the DLS achieved a 5-year disease-specific survival AUC of 0.70 (95%CI 0.66-0.73) and 0.69 (95%CI 0.64-0.72), and added significant predictive value to a set of 9 clinicopathologic features. To interpret the DLS, we explored the ability of different human-interpretable features to explain the variance in DLS scores. We observed that clinicopathologic features such as T-category, N-category, and grade explained a small fraction of the variance in DLS scores (R2=18% in both validation sets). Next, we generated human-interpretable histologic features by clustering embeddings from a deep-learning based image-similarity model and showed that they explain the majority of the variance (R2 of 73% to 80%). Furthermore, the clustering-derived feature most strongly associated with high DLS scores was also highly prognostic in isolation. With a distinct visual appearance (poorly differentiated tumor cell clusters adjacent to adipose tissue), this feature was identified by annotators with 87.0-95.5% accuracy. Our approach can be used to explain predictions from a prognostic deep learning model and uncover potentially-novel prognostic features that can be reliably identified by people for future validation studies.