Masked Omics Modeling for Multimodal Representation Learning across Histopathology and Molecular Profiles
Not on the map. Why?
Abstract (arXiv)
Self-supervised learning (SSL) has driven major advances in computational pathology by enabling the learning of rich representations from histopathology data. Yet, tissue analysis alone may fall short in capturing broader molecular complexity, as key complementary information resides in high-dimensional omics profiles such as transcriptomics, methylomics, and genomics. To address this gap, we introduce MORPHEUS, the first multimodal pre-training strategy that integrates histopathology images and multi-omics data within a shared transformer-based architecture. At its core, MORPHEUS relies on a novel masked omics modeling objective that encourages the model to learn meaningful cross-modal relationships. This yields a general-purpose pre-trained encoder that can be applied to histopathology alone or in combination with any subset of omics modalities. Beyond inference, MORPHEUS also supports flexible any-to-any omics reconstruction, enabling one or more omics profiles to be reconstructed from any modality subset that includes histopathology. Pre-trained on a large pan-cancer cohort, MORPHEUS shows substantial improvements over supervised and SSL baselines across diverse tasks and modality combinations. Together, these capabilities position it as a promising direction for the development of multimodal foundation models in oncology. Code is publicly available at https://github.com/Lucas-rbnt/MORPHEUS
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 52 references that are in the map. First posted on arXiv 1 Aug 2025.
Builds on
29 of 52 references.
References · 29
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2017Deep Setscited by 1,421
- 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
- 2018Attention-based Deep Multiple Instance Learningcited by 1,095
- 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
- 2017Self-Normalizing Neural Networkscited by 598
- 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 555
- 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 359
17 more
- 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
- 2019Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosiscited by 165
- 2020Self supervised contrastive learning for digital histopathologycited by 146
- 20234M: Massively Multimodal Masked Modelingcited by 120
- 2023Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Predictioncited by 92
- 2023Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Predictioncited by 86
- 2018A Scalable Discrete-Time Survival Model for Neural Networkscited by 72
- 2024Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathologycited by 72
- 2019Continuous and Discrete-Time Survival Prediction with Neural Networkscited by 70
- 2025Molecular-driven Foundation Model for Oncologic Pathologycited by 55
- 2024Multimodal Prototyping for cancer survival predictioncited by 47
- 2024MoME: Mixture of Multimodal Experts for Cancer Survival Predictioncited by 23
- 2022Stability of clinical prediction models developed using statistical or machine learning methodscited by 12
- 2023Masked Pre-Training of Transformers for Histology Image Analysiscited by 12
- 2024DRIM: Learning Disentangled Representations from Incomplete Multimodal Healthcare Datacited by 10
- 2025Self-Supervision Enhances Instance-based Multiple Instance Learning Methods in Digital Pathology: A Benchmark Studycited by 6
- 2024Uncovering the Genetic Basis of Glioblastoma Heterogeneity through Multimodal Analysis of Whole Slide Images and RNA Sequencing Datacited by 1
Built on by
Not cited yet.