Trimodal Glioma Representation Alignment via Volumetric Contrastive Learning
Not on the map. Why?
Abstract (arXiv)
Glioma grading and survival prediction require the integration of heterogeneous information collected at different spatial and biological scales. Histopathology describes tissue morphology, mRNA expression captures molecular activity, and magnetic resonance imaging provides a non-invasive view of tumor extent and radiological heterogeneity. Existing glioma prognosis models often combine only two of these sources, while their alignment objectives remain mostly pairwise. This paper introduces GLORIA, a novel trimodal framework for GLioma Omics - Radiology - hIstopathology Alignment. GLORIA processes whole-slide image regions, gene-expression profiles, and 3D MRI volumes through modality-specific encoders, projects them into a shared latent space, and aligns them with a Gramian contrastive loss that measures the volume spanned by the three modality embeddings. The aligned representations are fused through a cross-modal gating module and optimized jointly for three-class glioma grading and overall survival prediction. We evaluate GLORIA on a matched TCGA-GBM/LGG and BraTS21 cohort, comprising 132 patients with all three modalities. On the shared trimodal test set, GLORIA improves over the bimodal WSI-mRNA baseline in all the metrics considered.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 12 Jun 2026.
Builds on
16 of 32 references.
References · 16
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
- 2018Attention-based Deep Multiple Instance Learningcited by 1,095
- 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
- 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
- 2021The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classificationcited by 518
- 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 370
- 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 169
- 2019Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosiscited by 165
- 2024Gramian Multimodal Representation Learning and Alignmentcited by 54
- 2021Deep Orthogonal Fusion: Multimodal Prognostic Biomarker Discovery Integrating Radiology, Pathology, Genomic, and Clinical Datacited by 29
4 more
- 2022Survival Prediction of Brain Cancer with Incomplete Radiology, Pathology, Genomics, and Demographic Datacited by 25
- 2025A TRIANGLE Enables Multimodal Alignment Beyond Cosine Similaritycited by 18
- 2024Comprehensive Multimodal Deep Learning Survival Prediction Enabled by a Transformer Architecture: A Multicenter Study in Glioblastomacited by 9
- 2026Closing the gap in multimodal medical representation alignmentcited by 1
Built on by
Not cited yet.