# Predicting molecular phenotypes from histopathology images: a transcriptome-wide expression-morphology analysis in breast cancer

arXiv: https://arxiv.org/abs/2009.08917 · HTML: https://mlsanity.com/paper/2009.08917/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2020.

## Built on by (most cited first, 21 of 21)

- [DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2203.12081/) (2022)
- [Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning](https://mlsanity.com/paper/2306.01859/) (2023)
- [Attention-Challenging Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2311.07125/) (2023)
- [The ACROBAT 2022 Challenge: Automatic Registration Of Breast Cancer Tissue](https://mlsanity.com/paper/2305.18033/) (2023)
- [Domain-specific optimization and diverse evaluation of self-supervised models for histopathology](https://mlsanity.com/paper/2310.13259/) (2023)
- [ACROBAT -- a multi-stain breast cancer histological whole-slide-image data set from routine diagnostics for computational pathology](https://mlsanity.com/paper/2211.13621/) (2022)
- [hist2RNA: An efficient deep learning architecture to predict gene expression from breast cancer histopathology images](https://mlsanity.com/paper/2304.04507/) (2023)
- [Transcriptome-wide prediction of prostate cancer gene expression from histopathology images using co-expression based convolutional neural networks](https://mlsanity.com/paper/2104.09310/) (2021)
- [Evaluating Computational Pathology Foundation Models for Prostate Cancer Grading under Distribution Shifts](https://mlsanity.com/paper/2410.06723/) (2024)
- [Deep Learning-Based Prediction of Molecular Tumor Biomarkers from H&E: A Practical Review](https://mlsanity.com/paper/2211.14847/) (2022)
- [Increasing the usefulness of already existing annotations through WSI registration](https://mlsanity.com/paper/2303.06727/) (2023)
- [Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathology](https://mlsanity.com/paper/2603.08328/) (2026)
- [Domain-specific transfer learning in the automated scoring of tumor-stroma ratio from histopathological images of colorectal cancer](https://mlsanity.com/paper/2212.14652/) (2022)
- [A Mixture of Experts (MoE) model to improve AI-based computational pathology prediction performance under variable levels of histopathology image blur](https://mlsanity.com/paper/2405.09298/) (2024)
- [Distilling High Diagnostic Value Patches for Whole Slide Image Classification Using Attention Mechanism](https://mlsanity.com/paper/2407.19821/) (2024)
- [What makes for good morphology representations for spatial omics?](https://mlsanity.com/paper/2407.20660/) (2024)
- [HistoSPACE: Histology-Inspired Spatial Transcriptome Prediction And Characterization Engine](https://mlsanity.com/paper/2408.03592/) (2024)
- [Evaluation and Prognostic Validation of Deep Regression Models for WSI-Based Gene-Expression Prediction](https://mlsanity.com/paper/2410.00945/) (2024)
- [Structural Prognostic Event Modeling for Multimodal Cancer Survival Analysis](https://mlsanity.com/paper/2512.01116/) (2025)
- [ProtoPathway: Biologically Structured Prototype-Pathway Fusion for Multimodal Cancer Survival Prediction](https://mlsanity.com/paper/2605.21454/) (2026)
- [JASPR: Joint Spatial Representation learning of histology and spatial genomics for improved virtual genomic screening and clinical prognostication](https://mlsanity.com/paper/2606.28395/) (2026)

## Abstract (arXiv)

Molecular phenotyping is central in cancer precision medicine, but remains costly and standard methods only provide a tumour average profile. Microscopic morphological patterns observable in histopathology sections from tumours are determined by the underlying molecular phenotype and associated with clinical factors. The relationship between morphology and molecular phenotype has a potential to be exploited for prediction of the molecular phenotype from the morphology visible in histopathology images. We report the first transcriptome-wide Expression-MOrphology (EMO) analysis in breast cancer, where gene-specific models were optimised and validated for prediction of mRNA expression both as a tumour average and in spatially resolved manner. Individual deep convolutional neural networks (CNNs) were optimised to predict the expression of 17,695 genes from hematoxylin and eosin (HE) stained whole slide images (WSIs). Predictions for 9,334 (52.75%) genes were significantly associated with RNA-sequencing estimates (FDR adjusted p-value < 0.05). 1,011 of the genes were brought forward for validation, with 876 (87%) and 908 (90%) successfully replicated in internal and external test data, respectively. Predicted spatial intra-tumour variabilities in expression were validated in 76 genes, out of which 59 (77.6%) had a significant association (FDR adjusted p-value < 0.05) with spatial transcriptomics estimates. These results suggest that the proposed methodology can be applied to predict both tumour average gene expression and intra-tumour spatial expression directly from morphology, thus providing a scalable approach to characterise intra-tumour heterogeneity.