# Transcriptome-informed multi-modal AI for predicting neoadjuvant therapy response from breast cancer biopsies

arXiv: https://arxiv.org/abs/2610.03693 · HTML: https://mlsanity.com/paper/2610.03693/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 111 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Builds on (most cited first, 13 of 13)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Artificial Intelligence for Digital and Computational Pathology](https://mlsanity.com/paper/2401.06148/) (2023)
- [Molecular-driven Foundation Model for Oncologic Pathology](https://mlsanity.com/paper/2501.16652/) (2025)
- [Transcriptomics-guided Slide Representation Learning in Computational Pathology](https://mlsanity.com/paper/2405.11618/) (2024)
- [Generating crossmodal gene expression from cancer histopathology improves multimodal AI predictions](https://mlsanity.com/paper/2502.00568/) (2025)
- [Teaching pathology foundation models to accurately predict gene expression with parameter efficient knowledge transfer](https://mlsanity.com/paper/2504.07061/) (2025)
- [Multi-modal AI for comprehensive breast cancer prognostication](https://mlsanity.com/paper/2410.21256/) (2024)
- [Causal multi-modal AI for personalized chemosensitivity prediction](https://mlsanity.com/paper/2609.13567/) (2026)
- [The Most Important Features in Generalized Additive Models Might Be Groups of Features](https://mlsanity.com/paper/2506.19937/) (2025)

## Abstract (arXiv)

Scarcity of labeled data limits development of deep learning biomarkers in oncology. We develop a two-stage AI model predicting pathological complete response (pCR) to neoadjuvant therapy in breast cancer. The first stage learns the transcriptome from histopathology using 8,742 patients across 32 cancer types, corroborated by pathologist review and spatial agreement with measured expression. This simplifies the second stage to predicting pCR from inferred expression and clinical variables. Developed using 1,080 patients (five cohorts) and evaluated in 1,412 patients (nine cohorts), the model achieves a pooled AUROC of 0.79 (95% CI, 0.73-0.85), discriminating responders within molecular subtypes. It outperforms histopathological biomarkers, remaining stable across intratumoral sampling and with minimal biopsy tissue. Ablations show transcriptome-wide inference improves discrimination over clinical variables alone or one-stage pathology models, and robustness by avoiding genomic assays' gene selection constraints. These results indicate that biologically informed compression may generalize to data-sparse applications in precision oncology.