Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis
Not on the map. Why?
Abstract (arXiv)
Modern medicine relies on heterogeneous data sources spanning radiology, pathology, text reports, and structured clinical information. However, real-world patient data are frequently incomplete, with missing or sparsely acquired modalities, limiting the effectiveness of standard multimodal fusion approaches. To this end, we propose the Multimodal Flexible Redundancy-aware decomposed GAted Learning (Multi-FRuGaL) framework, a decomposition-aware, adaptive gated intermediate-fusion framework that performs modality-level representation learning under missing data. Multi-FRuGaL integrates per-modality encoders with a signal decomposition layer, an input-conditioned gating network, and an information-aware fusion objective to separate redundant from modality-specific complementary signals, selectively upweighting informative modalities and suppressing redundant or noisy inputs, and remaining well-defined even when multiple modalities are absent. We evaluate Multi-FRuGaL on two multimodal head and neck cancer cohorts: the HANCOCK challenge dataset (N = 763) comprising five modalities and two prognostic endpoints (5-year survival and 2-year recurrence), and the HECKTOR challenge dataset (N = 588) comprising three modalities for human papillomavirus (HPV) status classification. Multi-FRuGaL consistently achieves higher mean performance than the evaluated baselines across multiple tasks, improving AUC from 0.601 to 0.8496 for survival, from 0.672 to 0.8102 for recurrence, and achieving 0.975 AUC for HPV prediction on HECKTOR. For survival analysis, it further achieves a concordance index of 0.6814 for overall survival, 0.7421 for recurrence-free survival, and 0.7143 for progression-free survival on HANCOCK, and 0.7203 for recurrence-free survival on HECKTOR. Qualitative analyses further show that Multi-FRuGaL learns discriminative and robust multimodal representations, even under severe missing-modality conditions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 75 references that are in the map. First posted on arXiv 5 Jun 2026.
Builds on
35 of 75 references.
References · 35
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2017Squeeze-and-Excitation Networkscited by 4,789
- 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
- 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
- 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
- 2019Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosiscited by 165
- 2019Deep neural network models for computational histopathology: A surveycited by 165
- 2024Virchow2: Scaling Self-Supervised Mixed Magnification Models in Pathologycited by 161
- 2020Self supervised contrastive learning for digital histopathologycited by 146
23 more
- 2024Multimodal Alignment and Fusion: A Surveycited by 74
- 2022Integrated multimodal artificial intelligence framework for healthcare applicationscited by 71
- 2024Benchmarking foundation models as feature extractors for weakly-supervised computational pathologycited by 49
- 2024Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projectioncited by 46
- 2025ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Modelscited by 40
- 2022Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Reviewcited by 37
- 2025Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networkscited by 37
- 2020On the Benefits of Early Fusion in Multimodal Representation Learningcited by 27
- 2024A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applicationscited by 25
- 2024COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-trainingcited by 15
- 2025MIMO: A medical vision language model with visual referring multimodal input and pixel grounding multimodal outputcited by 15
- 2025STiL: Semi-supervised Tabular-Image Learning for Comprehensive Task-Relevant Information Exploration in Multimodal Classificationcited by 12
- 2023LANISTR: Multimodal Learning from Structured and Unstructured Datacited by 11
- 2025Distilled Prompt Learning for Incomplete Multimodal Survival Predictioncited by 11
- 2023Missing-modality Enabled Multi-modal Fusion Architecture for Medical Datacited by 8
- 2022CustOmics: A versatile deep-learning based strategy for multi-omics integrationcited by 6
- 2024HoneyBee: A Scalable Modular Framework for Creating Multimodal Oncology Datasets with Foundational Embedding Modelscited by 6
- 2023Asynchronous Events-based Panoptic Segmentation using Graph Mixer Neural Networkcited by 3
- 2025ModalTune: Fine-Tuning Slide-Level Foundation Models with Multi-Modal Information for Multi-task Learning in Digital Pathologycited by 2
- 2025PROFUSEme: PROstate Cancer Biochemical Recurrence Prediction via FUSEd Multi-modal Embeddingscited by 2
- 2024Multimodal Integration of Longitudinal Noninvasive Diagnostics for Survival Prediction in Immunotherapy Using Deep Learningcited by 1
- 2025impuTMAE: Multi-modal Transformer with Masked Pre-training for Missing Modalities Imputation in Cancer Survival Predictioncited by 1
- 2026PGcGAN: Pathological Gait-Conditioned GAN for Human Gait Synthesiscited by 1
Built on by
Not cited yet.