# A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction

arXiv: https://arxiv.org/abs/2604.03630 · HTML: https://mlsanity.com/paper/2604.03630/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 75 references that are in the map.
- First posted on arXiv 4 Apr 2026.

## Builds on (most cited first, 7 of 7)

- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning](https://mlsanity.com/paper/2306.01859/) (2023)
- [Exemplar Guided Deep Neural Network for Spatial Transcriptomics Analysis of Gene Expression Prediction](https://mlsanity.com/paper/2210.16721/) (2022)
- [Distilling foundation models for robust and efficient models in digital pathology](https://mlsanity.com/paper/2501.16239/) (2025)
- [Multimodal contrastive learning for spatial gene expression prediction using histology images](https://mlsanity.com/paper/2407.08216/) (2024)

## Built on by (most cited first, 3 of 3)

- [Bridging the Modality Bottleneck in Pathology MIL through Virtual Molecular Staining](https://mlsanity.com/paper/2605.16392/) (2026)
- [VOICE: A Vision-Omics Foundation Model Integrating Direct and Retrieval-Based Prediction of In-situ Single-Cell Gene Expression](https://mlsanity.com/paper/2608.08366/) (2026)
- [HierarchicalDAEW: Domain-Aware Edge-Weighted Graph Convolution with Evidential Uncertainty for Multi-Section Spatial Gene Expression Prediction from H&E Histology](https://mlsanity.com/paper/2607.20896/) (2026)

## Abstract (arXiv)

Spatial transcriptomics (ST) enables gene expression mapping within anatomical context but remains costly and low-throughput. Hematoxylin and eosin (H&E) staining offers rich morphology yet lacks molecular resolution. We present ours (Spatial Transcriptomics and histOlogy Representation Model), a foundation model trained on 1.2 million spatially resolved transcriptomic profiles with matched histology across 18 organs. Using a hierarchical architecture integrating morphological features, gene expression, and spatial context, STORM bridges imaging and omics through robust molecular--morphological representations. STORM enhances spatial domain discovery, producing biologically coherent tissue maps, and outperforms existing methods in predicting spatial gene expression from H&E images across 11 tumor types. The model is platform-agnostic, performing consistently across Visium, Xenium, Visium HD, and CosMx. Applied to 23 independent cohorts comprising 7,245 patients, STORM significantly improves immunotherapy response prediction and prognostication over established biomarkers, providing a scalable framework for spatially informed discovery and clinical precision medicine.