# Challenges and proposed solutions in modeling multimodal medical data: A systematic review

arXiv: https://arxiv.org/abs/2505.06945 · HTML: https://mlsanity.com/paper/2505.06945/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 130 references that are in the map.
- First posted on arXiv 11 May 2025.

## Builds on (most cited first, 26 of 26)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Multimodal Deep Learning](https://mlsanity.com/paper/2301.04856/) (2023)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2102.04830/) (2021)
- [Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling](https://mlsanity.com/paper/2307.14126/) (2023)
- [M3Care: Learning with Missing Modalities in Multimodal Healthcare Data](https://mlsanity.com/paper/2210.17292/) (2022)
- [Integrated multimodal artificial intelligence framework for healthcare applications](https://mlsanity.com/paper/2202.12998/) (2022)
- [A Variational Information Bottleneck Approach to Multi-Omics Data Integration](https://mlsanity.com/paper/2102.03014/) (2021)
- [Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generation](https://mlsanity.com/paper/2407.02235/) (2024)
- [A Survey of Multimodal Information Fusion for Smart Healthcare: Mapping the Journey from Data to Wisdom](https://mlsanity.com/paper/2306.11963/) (2023)
- [Multimodal Attention-based Deep Learning for Alzheimer's Disease Diagnosis](https://mlsanity.com/paper/2206.08826/) (2022)
- [Random Forest Missing Data Algorithms](https://mlsanity.com/paper/1701.05305/) (2017)
- [Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data](https://mlsanity.com/paper/1904.06236/) (2019)
- [Artificial Intelligence-Based Methods for Fusion of Electronic Health Records and Imaging Data](https://mlsanity.com/paper/2210.13462/) (2022)
- [Multimodal Subspace Support Vector Data Description](https://mlsanity.com/paper/1904.07698/) (2019)
- [Polynomial Matrix Completion for Missing Data Imputation and Transductive Learning](https://mlsanity.com/paper/1912.06989/) (2019)
- [TFormer: A throughout fusion transformer for multi-modal skin lesion diagnosis](https://mlsanity.com/paper/2211.11393/) (2022)
- [MRI-based Alzheimer's disease prediction via distilling the knowledge in multi-modal data](https://mlsanity.com/paper/2104.03618/) (2021)
- [Multiple Kernel Learning in the Primal for Multi-modal Alzheimer's Disease Classification](https://mlsanity.com/paper/1310.0890/) (2013)
- [CNN-LSTM Based Multimodal MRI and Clinical Data Fusion for Predicting Functional Outcome in Stroke Patients](https://mlsanity.com/paper/2205.05545/) (2022)
- [HoneyBee: A Scalable Modular Framework for Creating Multimodal Oncology Datasets with Foundational Embedding Models](https://mlsanity.com/paper/2405.07460/) (2024)
- [Multi-modal Disease Classification in Incomplete Datasets Using Geometric Matrix Completion](https://mlsanity.com/paper/1803.11550/) (2018)
- [Multimodal Approaches to Fair Image Classification: An Ethical Perspective](https://mlsanity.com/paper/2412.12165/) (2024)

## Abstract (arXiv)

Multimodal data modeling has emerged as a powerful approach in clinical research, enabling the integration of diverse data types such as imaging, genomics, wearable sensors, and electronic health records. Despite its potential to improve diagnostic accuracy and support personalized care, modeling such heterogeneous data presents significant technical challenges. This systematic review synthesizes findings from 69 studies to identify common obstacles, including missing modalities, limited sample sizes, dimensionality imbalance, interpretability issues, and finding the optimal fusion techniques. We highlight recent methodological advances, such as transfer learning, generative models, attention mechanisms, and neural architecture search that offer promising solutions. By mapping current trends and innovations, this review provides a comprehensive overview of the field and offers practical insights to guide future research and development in multimodal modeling for medical applications.