mlsanityData to 14 Oct 2026

Multimodal Fusion on Low-quality Data: A Comprehensive Survey

arXiv 2404.18947 ↗First posted 27 Apr 2024cs.LG cs.AIcited by 49

Not on the map. Why?

Abstract (arXiv)

Multimodal fusion focuses on integrating information from multiple modalities with the goal of more accurate prediction, which has achieved remarkable progress in a wide range of scenarios, including autonomous driving and medical diagnosis. However, the reliability of multimodal fusion remains largely unexplored especially under low-quality data settings. This paper surveys the common challenges and recent advances of multimodal fusion in the wild and presents them in a comprehensive taxonomy. From a data-centric view, we identify four main challenges that are faced by multimodal fusion on low-quality data, namely (1) noisy multimodal data that are contaminated with heterogeneous noises, (2) incomplete multimodal data that some modalities are missing, (3) imbalanced multimodal data that the qualities or properties of different modalities are significantly different and (4) quality-varying multimodal data that the quality of each modality dynamically changes with respect to different samples. This new taxonomy will enable researchers to understand the state of the field and identify several potential directions. We also provide discussion for the open problems in this field together with interesting future research directions.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Builds on 73 of its 175 references that are in the map. First posted on arXiv 27 Apr 2024.

Builds on

73 of 175 references.

References · 73

  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  2. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  3. 2017The Kinetics Human Action Video Datasetcited by 1,909
  4. 2018SlowFast Networks for Video Recognitioncited by 1,867
  5. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  6. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  7. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  8. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  9. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 648
  10. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  11. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  12. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
38 more
  1. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
  2. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 264
  3. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 215
  4. 2019Addressing Failure Prediction by Learning Model Confidencecited by 186
  5. 2018DenseFuse: A Fusion Approach to Infrared and Visible Imagescited by 180
  6. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 153
  7. 2013A Survey on Multi-view Learningcited by 152
  8. 2020Audiovisual SlowFast Networks for Video Recognitioncited by 146
  9. 2022Multimodal Token Fusion for Vision Transformerscited by 136
  10. 2019MMTM: Multimodal Transfer Module for CNN Fusioncited by 129
  11. 2022Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)cited by 122
  12. 2020Deep Multimodal Fusion by Channel Exchangingcited by 118
  13. 2022PMR: Prototypical Modal Rebalance for Multimodal Learningcited by 116
  14. 2019Multimodal End-to-End Autonomous Drivingcited by 110
  15. 2022Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networkscited by 95
  16. 2021Trusted Multi-View Classificationcited by 94
  17. 2023Filtering, Distillation, and Hard Negatives for Vision-Language Pre-Trainingcited by 91
  18. 2020Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problemscited by 83
  19. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 82
  20. 2023Multimodal Representation Learning by Alternating Unimodal Adaptationcited by 75
  21. 2021Multimodal Object Detection via Probabilistic Ensemblingcited by 68
  22. 2015Multimodal Deep Learning for Robust RGB-D Object Recognitioncited by 57
  23. 2018Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detectioncited by 57
  24. 2022Dynamic Multimodal Fusioncited by 55
  25. 2022Towards Good Practices for Missing Modality Robust Action Recognitioncited by 54
  26. 2023On Uni-Modal Feature Learning in Supervised Multi-Modal Learningcited by 53
  27. 2024DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistencycited by 53
  28. 2023Enhancing multimodal cooperation via sample-level modality valuationcited by 44
  29. 2023Understanding Multimodal Contrastive Learning and Incorporating Unpaired Datacited by 43
  30. 2019Doubly Aligned Incomplete Multi-view Clusteringcited by 40
  31. 2021Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributionscited by 39
  32. 2021AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognitioncited by 36
  33. 2017VIGAN: Missing View Imputation with Generative Adversarial Networkscited by 33
  34. 2019Noise-Aware Unsupervised Deep Lidar-Stereo Fusioncited by 31
  35. 2022A Survey on Incomplete Multi-view Clusteringcited by 31
  36. 2022Uncertainty-Aware Multi-View Representation Learningcited by 29
  37. 2023MMCosine: Multi-Modal Cosine Loss Towards Balanced Audio-Visual Fine-Grained Learningcited by 28
  38. 2019One-Pass Incomplete Multi-view Clusteringcited by 24

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 17; 2026: 23.

49 papers.

Sort:

By area:

  1. 2024MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistancecited by 73
  2. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  3. 2024DeMo: Decoupled Feature-Based Mixture of Experts for Multi-Modal Object Re-Identificationcited by 29
  4. 2025IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identificationcited by 28
  5. 2025Benchmarking Multi-modal Semantic Segmentation under Sensor Failures: Missing and Noisy Modality Robustnesscited by 18
  6. 2025Improving Multimodal Learning via Imbalanced Learningcited by 17
  7. 2025DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learningcited by 16
  8. 2025Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospectscited by 11
  9. 2024MM-GTUNets: Unified Multi-Modal Graph Deep Learning for Brain Disorders Predictioncited by 9
  10. 2025Improving Multimodal Learning Balance and Sufficiency through Data Remixingcited by 8
  11. 2025Fusion Meets Diverse Conditions: A High-diversity Benchmark and Baseline for UAV-based Multimodal Object Detection with Condition Cuescited by 6
  12. 2024RSEA-MVGNN: Multi-View Graph Neural Network with Reliable Structural Enhancement and Aggregationcited by 4
37 more
  1. 2024Dynamic Evidence Decoupling for Trusted Multi-view Learningcited by 4
  2. 2026Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classificationcited by 4
  3. 2025Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbationcited by 3
  4. 2025NSF-MAP: Neurosymbolic Multimodal Fusion for Robust and Interpretable Anomaly Prediction in Assembly Pipelinescited by 3
  5. 2025A Novel Multi-layer Task-centric and Data Quality Framework for Autonomous Drivingcited by 2
  6. 2025Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learningcited by 2
  7. 2025MIDAS: Misalignment-based Data Augmentation Strategy for Imbalanced Multimodal Learningcited by 2
  8. 2026SEF-MAP: Subspace-Decomposed Expert Fusion for Robust Multimodal HD Map Predictioncited by 2
  9. 2024Multi-view biomedical foundation models for molecule-target and property predictioncited by 1
  10. 2025RollingQ: Reviving the Cooperation Dynamics in Multimodal Transformercited by 1
  11. 2025Complementarity-driven Representation Learning for Multi-modal Knowledge Graph Completioncited by 1
  12. 2025MULTIBENCH++: A Unified and Comprehensive Multimodal Fusion Benchmarking Across Specialized Domainscited by 1
  13. 2026ModalImmune: Immunity Driven Unlearning via Self Destructive Trainingcited by 1
  14. 2026REMIND: Rethinking Medical High-Modality Learning under Missingness--A Long-Tailed Distribution Perspectivecited by 1
  15. 2026Multimodal Mixture-of-Experts with Retrieval Augmentation for Protein Active Site Identificationcited by 1
  16. 2026BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Ratescited by 1
  17. 2026Multi-Perspective Evidence Synthesis and Reasoning for Unsupervised Multimodal Entity Linkingcited by 1
  18. 2024Enhancing Multimodal Medical Image Classification using Cross-Graph Modal Contrastive Learningcited by 0
  19. 2024Discrepancy-Aware Attention Network for Enhanced Audio-Visual Zero-Shot Learningcited by 0
  20. 2024Multi-QuAD: Multi-Level Quality-Adaptive Dynamic Network for Reliable Multimodal Classificationcited by 0
  21. 2025REACT-KD: Region-Aware Cross-modal Topological Knowledge Distillation for Interpretable Medical Image Classificationcited by 0
  22. 2026Test-time Adaptive Hierarchical Co-enhanced Denoising Network for Reliable Multimodal Classificationcited by 0
  23. 2026Bayesian Integration of Nonlinear Incomplete Clinical Datacited by 0
  24. 2026Plug, Play, and Fortify: A Low-Cost Module for Robust Multimodal Image Understanding Modelscited by 0
  25. 2026Modeling and Measuring Redundancy in Multisource Multimodal Data for Autonomous Drivingcited by 0
  26. 2026Mind the Gap: A Framework for Assessing Pitfalls in Multimodal Active Learningcited by 0
  27. 2026BLEG: LLM Functions as Powerful fMRI Graph-Enhancer for Brain Network Analysiscited by 0
  28. 2026Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibrationcited by 0
  29. 2026Beyond Semantics: An Evidential Reasoning-Aware Multi-View Learning Framework for Trustworthy Mental Health Predictioncited by 0
  30. 2026Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognitioncited by 0
  31. 2026Robust Fuzzy Multi-view Learning under View Conflictcited by 0
  32. 2026DarkVGGT: Seeing Through Darkness Using Thermal Geometrycited by 0
  33. 2026VIS-DICT: A Visual Dictionary for Missing Modality Imputation in Social Network Depression Detectioncited by 0
  34. 2026Where Should Language Sit in a Multimodal Model? Lessons from What Language Does to Human Perception and Cognitioncited by 0
  35. 2026Fusion Anything: A Generalized Multimodal Foundation Modelcited by 0
  36. 2026Knowing When to Trust Images: Reliability-Aware Multi-modal Entity Alignmentcited by 0
  37. 2026GeoBalance: Geometry-Aware Monitoring and Reconstruction with Asymmetric Optimization for Balanced Multimodal Learningcited by 0