# Bridging the Modality Bottleneck in Pathology MIL through Virtual Molecular Staining

arXiv: https://arxiv.org/abs/2605.16392 · HTML: https://mlsanity.com/paper/2605.16392/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Multimodal Whole Slide Foundation Model for Pathology](https://mlsanity.com/paper/2411.19666/) (2024)
- [BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images](https://mlsanity.com/paper/2111.04740/) (2021)
- [HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis](https://mlsanity.com/paper/2406.16192/) (2024)
- [Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology](https://mlsanity.com/paper/2405.11643/) (2024)
- [Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology](https://mlsanity.com/paper/2402.17228/) (2024)
- [Multimodal Prototyping for cancer survival prediction](https://mlsanity.com/paper/2407.00224/) (2024)
- [Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides](https://mlsanity.com/paper/2112.02222/) (2021)
- [Handcrafted Histological Transformer (H2T): Unsupervised Representation of Whole Slide Images](https://mlsanity.com/paper/2202.07001/) (2022)
- [Evidential time-to-event prediction with calibrated uncertainty quantification](https://mlsanity.com/paper/2411.07853/) (2024)
- [DPsurv: Dual-Prototype Evidential Fusion for Uncertainty-Aware and Interpretable Whole-Slide Image Survival Prediction](https://mlsanity.com/paper/2510.00053/) (2025)
- [An evidential time-to-event prediction model based on Gaussian random fuzzy numbers](https://mlsanity.com/paper/2406.13487/) (2024)
- [Mixture of Mini Experts: Overcoming the Linear Layer Bottleneck in Multiple Instance Learning](https://mlsanity.com/paper/2603.22198/) (2026)
- [SPADE: Spatial Transcriptomics and Pathology Alignment Using a Mixture of Data Experts for an Expressive Latent Space](https://mlsanity.com/paper/2506.21857/) (2025)
- [Towards Spatial Transcriptomics-driven Pathology Foundation Models](https://mlsanity.com/paper/2602.14177/) (2026)
- [A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction](https://mlsanity.com/paper/2604.03630/) (2026)
- [MINT: Molecularly Informed Training with Spatial Transcriptomics Supervision for Pathology Foundation Models](https://mlsanity.com/paper/2603.07895/) (2026)

## Built on by (most cited first, 2 of 2)

- [Semantic-Anchored Evidential Fusion for Domain-Robust Whole-Slide Survival Analysis](https://mlsanity.com/paper/2606.19966/) (2026)
- [Evidential Fusion Network for Multimodal Survival Prediction under Missing Modalities](https://mlsanity.com/paper/2606.20757/) (2026)

## Abstract (arXiv)

Multiple instance learning (MIL) is the dominant framework for whole-slide image analysis in computational pathology, typically combining a frozen patch encoder, a projection layer, and a slide-level aggregator. While encoders and aggregators have been extensively studied, the projection layer remains a largely morphology-only bottleneck. This limits endpoints such as biomarker status and survival, which are governed by a molecular state that is not fully captured by H&E morphology. We introduce Molecularly Informed Staining Transform (MIST), a plug-in replacement for the MIL projection layer that uses paired spatial transcriptomics only during training to construct virtual molecular stains. MIST clusters gene expression profiles into cross-modal prototypes, anchors them in the frozen foundation model feature space, and uses them to reorganize H&E patch features along molecularly guided axes. It requires no transcriptomics at inference and can be inserted before standard MIL aggregators. We evaluate MIST across 23 downstream tasks and 8 MIL aggregators. MIST improves 240 of 256 configurations over the standard projection layer, with an average gain of +3.5%, observed consistently across endpoint types: +5.2% on survival prediction, +3.3% on tissue subtyping, and +2.6% on biomarker prediction. Ablations confirm that gene-derived prototypes are the primary source of the gains, while spatial, biological, and pathological analyses show that cross-modal prototype affinities capture spatially coherent molecular programs from H&E alone.