Data-Efficient Multimodal Alignment for Histopathology-based Molecular Prediction
Not on the map. Why?
Abstract (arXiv)
H&E-stained whole-slide images offer cohort-scale availability and rich spatial context but lack molecular specificity, whereas bulk RNA-seq provides transcriptome-wide resolution at high cost with limited archival availability. We show that training a lightweight alignment module atop frozen histopathology and RNA-Seq foundation models enables open-vocabulary molecular prompting -- querying H&E slides with gene-set signatures to predict pathway activity without sequencing or end-to-end retraining. Using contrastive learning on a multi-cancer cohort (N=1,720), we achieve a 25-fold improvement in retrieval over baseline methods. Systematic analysis reveals a graduated predictability spectrum: morphologically grounded programs (cell-cycle programs, immune-related) are most reliably predicted (R^2>0.5), while predicting pathways with no morphological footprint remains challenging as expected. We validate clinical utility on the POSEIDON clinical trial: H&E-predicted squamous cell carcinoma scores recapitulate NSCLC subtype identity and predicted IFN-gamma mirror PD-L1 tumor-cell expression groups. Furthermore, genesets describing immune activation and fibrosis predict known tumor microenvironment archetypes from histology alone. We further validate generalization of our approach across unseen cohorts and demonstrate data-efficient domain adaptation, establishing a slide-native framework for molecular analysis on H&E images.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 5 of its 23 references that are in the map. First posted on arXiv 29 Jun 2026.
Builds on
5 of 23 references.
References · 5
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2024Multimodal Whole Slide Foundation Model for Pathologycited by 156
- 2025Molecular-driven Foundation Model for Oncologic Pathologycited by 52
- 2024Screen Them All: High-Throughput Pan-Cancer Genetic and Phenotypic Biomarker Screening from H&E Whole Slide Imagescited by 8
- 2026HistoPrism: Unlocking Functional Pathway Analysis from Pan-Cancer Histology via Gene Expression Predictioncited by 5
Built on by
Not cited yet.