# Learning to Predict RNA Sequence Expressions from Whole Slide Images with Applications for Search and Classification

arXiv: https://arxiv.org/abs/2203.13997 · HTML: https://mlsanity.com/paper/2203.13997/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 26 Mar 2022.

## Builds on (most cited first, 10 of 10)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification](https://mlsanity.com/paper/1504.07947/) (2015)
- [Automated Gleason Grading of Prostate Biopsies using Deep Learning](https://mlsanity.com/paper/1907.07980/) (2019)
- [Neural Ranking Models with Weak Supervision](https://mlsanity.com/paper/1704.08803/) (2017)
- [Yottixel -- An Image Search Engine for Large Archives of Histopathology Whole Slide Images](https://mlsanity.com/paper/1911.08748/) (2019)
- [Pan-Cancer Diagnostic Consensus Through Searching Archival Histopathology Images Using Artificial Intelligence](https://mlsanity.com/paper/1911.08736/) (2019)
- [A Closer Look at Domain Shift for Deep Learning in Histopathology](https://mlsanity.com/paper/1909.11575/) (2019)
- [Self-Supervised Similarity Learning for Digital Pathology](https://mlsanity.com/paper/1905.08139/) (2019)
- [Supervision and Source Domain Impact on Representation Learning: A Histopathology Case Study](https://mlsanity.com/paper/2005.08629/) (2020)

## Built on by (most cited first, 14 of 14)

- [HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis](https://mlsanity.com/paper/2406.16192/) (2024)
- [SEPAL: Spatial Gene Expression Prediction from Local Graphs](https://mlsanity.com/paper/2309.01036/) (2023)
- [On Image Search in Histopathology](https://mlsanity.com/paper/2401.08699/) (2024)
- [From Classical Machine Learning to Emerging Foundation Models: Review on Multimodal Data Integration for Cancer Research](https://mlsanity.com/paper/2507.09028/) (2025)
- [A Preliminary Investigation into Search and Matching for Tumour Discrimination in WHO Breast Taxonomy Using Deep Networks](https://mlsanity.com/paper/2308.11162/) (2023)
- [CIMIL-CRC: a clinically-informed multiple instance learning framework for patient-level colorectal cancer molecular subtypes classification from H&E stained images](https://mlsanity.com/paper/2401.16131/) (2024)
- [Selection of Distinct Morphologies to Divide & Conquer Gigapixel Pathology Images](https://mlsanity.com/paper/2311.09902/) (2023)
- [Cell-Type Prototype-Informed Neural Network for Gene Expression Estimation from Pathology Images](https://mlsanity.com/paper/2603.18461/) (2026)
- [Multimodal Learning for Scalable Representation of High-Dimensional Medical Data](https://mlsanity.com/paper/2409.13115/) (2024)
- [Translating Imaging to Genomics: Leveraging Transformers for Predictive Modeling](https://mlsanity.com/paper/2408.00311/) (2024)
- [RNA-FM: Flow-Matching Generative Model for Genome-wide RNA-Seq Prediction](https://mlsanity.com/paper/2605.11622/) (2026)
- [SNR-ST-Mix: Sample-specific Neighborhood Regression Mixup for Augmented Spatial Transcriptomics Imputation with Deep Neural Network](https://mlsanity.com/paper/2606.08712/) (2026)
- [JASPR: Joint Spatial Representation learning of histology and spatial genomics for improved virtual genomic screening and clinical prognostication](https://mlsanity.com/paper/2606.28395/) (2026)
- [MoSPR: Histology-to-Gene Expression Prediction with Morpho-Spatial Macrostates and Low-Rank Molecular Programs](https://mlsanity.com/paper/2609.34280/) (2026)

## Abstract (arXiv)

Deep learning methods are widely applied in digital pathology to address clinical challenges such as prognosis and diagnosis. As one of the most recent applications, deep models have also been used to extract molecular features from whole slide images. Although molecular tests carry rich information, they are often expensive, time-consuming, and require additional tissue to sample. In this paper, we propose tRNAsfomer, an attention-based topology that can learn both to predict the bulk RNA-seq from an image and represent the whole slide image of a glass slide simultaneously. The tRNAsfomer uses multiple instance learning to solve a weakly supervised problem while the pixel-level annotation is not available for an image. We conducted several experiments and achieved better performance and faster convergence in comparison to the state-of-the-art algorithms. The proposed tRNAsfomer can assist as a computational pathology tool to facilitate a new generation of search and classification methods by combining the tissue morphology and the molecular fingerprint of the biopsy samples.