# A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applications

arXiv: https://arxiv.org/abs/2408.02686 · HTML: https://mlsanity.com/paper/2408.02686/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 121 references that are in the map.
- First posted on arXiv 2 Aug 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [Towards Explainable Artificial Intelligence](https://mlsanity.com/paper/1909.12072/) (2019)
- [Multimodal Intelligence: Representation Learning, Information Fusion, and Applications](https://mlsanity.com/paper/1911.03977/) (2019)
- [A review: Deep learning for medical image segmentation using multi-modality fusion](https://mlsanity.com/paper/2004.10664/) (2020)
- [Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies](https://mlsanity.com/paper/2010.10802/) (2020)
- [MedFuse: Multi-modal fusion with clinical time-series data and chest X-ray images](https://mlsanity.com/paper/2207.07027/) (2022)
- [Deep Learning in Bioinformatics](https://mlsanity.com/paper/1603.06430/) (2016)
- [A Review on Explainability in Multimodal Deep Neural Nets](https://mlsanity.com/paper/2105.07878/) (2021)
- [A Review on Methods and Applications in Multimodal Deep Learning](https://mlsanity.com/paper/2202.09195/) (2022)
- [Vision+X: A Survey on Multimodal Learning in the Light of Data](https://mlsanity.com/paper/2210.02884/) (2022)
- [Transformer-based Self-supervised Multimodal Representation Learning for Wearable Emotion Recognition](https://mlsanity.com/paper/2303.17611/) (2023)
- [Multimodal Explainability via Latent Shift applied to COVID-19 stratification](https://mlsanity.com/paper/2212.14084/) (2022)
- [Multi-objective optimization determines when, which and how to fuse deep networks: an application to predict COVID-19 outcomes](https://mlsanity.com/paper/2204.03772/) (2022)
- [A Deep Learning Approach for Virtual Contrast Enhancement in Contrast Enhanced Spectral Mammography](https://mlsanity.com/paper/2308.00471/) (2023)
- [Multimodal fusion of imaging and genomics for lung cancer recurrence prediction](https://mlsanity.com/paper/2002.01982/) (2020)
- [Multimodal Information Fusion for Glaucoma and DR Classification](https://mlsanity.com/paper/2209.00979/) (2022)
- [A graph neural network-based model with Out-of-Distribution Robustness for enhancing Antiretroviral Therapy Outcome Prediction for HIV-1](https://mlsanity.com/paper/2312.17506/) (2023)
- [Predicting Brain Degeneration with a Multimodal Siamese Neural Network](https://mlsanity.com/paper/2011.00840/) (2020)

## Built on by (most cited first, 25 of 25)

- [MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling](https://mlsanity.com/paper/2508.05492/) (2025)
- [MARIA: a Multimodal Transformer Model for Incomplete Healthcare Data](https://mlsanity.com/paper/2412.14810/) (2024)
- [XGeM: A Multi-Prompt Foundation Model for Multimodal Medical Data Generation](https://mlsanity.com/paper/2501.04614/) (2025)
- [Whole-Body Image-to-Image Translation for a Virtual Scanner in a Healthcare Digital Twin](https://mlsanity.com/paper/2503.15555/) (2025)
- [Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging](https://mlsanity.com/paper/2506.18434/) (2025)
- [Augmented Intelligence for Multimodal Virtual Biopsy in Breast Cancer Using Generative Artificial Intelligence](https://mlsanity.com/paper/2501.19176/) (2025)
- [Any-to-Any Vision-Language Model for Multimodal X-ray Imaging and Radiological Report Generation](https://mlsanity.com/paper/2505.01091/) (2025)
- [Lesion-Aware Generative Artificial Intelligence for Virtual Contrast-Enhanced Mammography in Breast Cancer](https://mlsanity.com/paper/2505.03018/) (2025)
- [Multimodal Representation Learning and Fusion](https://mlsanity.com/paper/2506.20494/) (2025)
- [Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer](https://mlsanity.com/paper/2505.01390/) (2025)
- [Multi-stage intermediate fusion for multimodal learning to classify non-small cell lung cancer subtypes from CT and PET](https://mlsanity.com/paper/2501.12425/) (2025)
- [MITS: A Large-Scale Multimodal Benchmark Dataset for Intelligent Traffic Surveillance](https://mlsanity.com/paper/2509.09730/) (2025)
- [Graph Coloring for Multi-Task Learning](https://mlsanity.com/paper/2509.16959/) (2025)
- [Compact Latent Manifold Translation: A Parameter-Efficient Foundation Model for Cross-Modal and Cross-Frequency Physiological Signal Synthesis](https://mlsanity.com/paper/2605.13248/) (2026)
- [Evaluating Vision Language Model Adaptations for Radiology Report Generation in Low-Resource Languages](https://mlsanity.com/paper/2505.01096/) (2025)
- [AudioFuse: Unified Spectral-Temporal Learning via a Hybrid ViT-1D CNN Architecture for Robust Phonocardiogram Classification](https://mlsanity.com/paper/2509.23454/) (2025)
- [Synergy vs. Noise: Performance-Guided Multimodal Fusion For Biochemical Recurrence-Free Survival in Prostate Cancer](https://mlsanity.com/paper/2511.11452/) (2025)
- [Longitudinal NSCLC Treatment Progression via Multimodal Generative Models](https://mlsanity.com/paper/2603.06147/) (2026)
- [LF2L: Loss Fusion Horizontal Federated Learning Across Heterogeneous Feature Spaces Using External Datasets Effectively: A Case Study in Second Primary Cancer Prediction](https://mlsanity.com/paper/2603.07249/) (2026)
- [Learning from Limited and Incomplete Data: A Multimodal Framework for Predicting Pathological Response in NSCLC](https://mlsanity.com/paper/2603.15100/) (2026)
- [Virtual Scanning for NSCLC Histology: Investigating the Discriminatory Power of Synthetic PET](https://mlsanity.com/paper/2605.02746/) (2026)
- [Resilient Vision-Tabular Multimodal Learning under Modality Missingness](https://mlsanity.com/paper/2605.12031/) (2026)
- [Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis](https://mlsanity.com/paper/2606.06867/) (2026)
- [Pulmonary Embolism Risk Stratification from CTPA and Medical Records: Vascular Graphs Are Not All You Need](https://mlsanity.com/paper/2606.25956/) (2026)
- [DAIF: A Data-Driven Intermediate Fusion Framework for Multimodal Supervised Learning via Approximate Message Passing](https://mlsanity.com/paper/2608.02769/) (2026)

## Abstract (arXiv)

Deep learning has revolutionized biomedical research by providing sophisticated methods to handle complex, high-dimensional data. Multimodal deep learning (MDL) further enhances this capability by integrating diverse data types such as imaging, textual data, and genetic information, leading to more robust and accurate predictive models. In MDL, differently from early and late fusion methods, intermediate fusion stands out for its ability to effectively combine modality-specific features during the learning process. This systematic review aims to comprehensively analyze and formalize current intermediate fusion methods in biomedical applications. We investigate the techniques employed, the challenges faced, and potential future directions for advancing intermediate fusion methods. Additionally, we introduce a structured notation to enhance the understanding and application of these methods beyond the biomedical domain. Our findings are intended to support researchers, healthcare professionals, and the broader deep learning community in developing more sophisticated and insightful multimodal models. Through this review, we aim to provide a foundational framework for future research and practical applications in the dynamic field of MDL.