mlsanityData to 7 Oct 2026

Deep Multimodal Learning with Missing Modality: A Survey

arXiv 2409.07825 ↗First posted 12 Sep 2024cs.CV cs.AI cs.LGcited by 89

Not on the map. Why?

Abstract (arXiv)

During multimodal model training and testing, certain data modalities may be absent due to sensor limitations, cost constraints, privacy concerns, or data loss, negatively affecting performance. Multimodal learning techniques designed to handle missing modalities can mitigate this by ensuring model robustness even when some modalities are unavailable. This survey reviews recent progress in Multimodal Learning with Missing Modality (MLMM), focusing on deep learning methods. It provides the first comprehensive survey that covers the motivation and distinctions between MLMM and standard multimodal learning setups, followed by a detailed analysis of current methods, applications, and datasets, concluding with challenges and future directions.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). Builds on 119 of its 275 references that are in the map. First posted on arXiv 12 Sep 2024.

Builds on

119 of 275 references.

References · 119

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  10. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  11. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  12. 2022Planning-oriented Autonomous Drivingcited by 1,068
38 more
  1. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  2. 2015Image-based Recommendations on Styles and Substitutescited by 702
  3. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  4. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  5. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
  6. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  7. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  8. 2020Deep Learning for Person Re-identification: A Survey and Outlookcited by 547
  9. 2020Rescaling Egocentric Visioncited by 529
  10. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 517
  11. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  12. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
  13. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  14. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  15. 2015Sketch-based Manga Retrieval using Manga109 Datasetcited by 326
  16. 20183D MRI brain tumor segmentation using autoencoder regularizationcited by 324
  17. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 243
  18. 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 213
  19. 2023Any-to-Any Generation via Composable Diffusioncited by 203
  20. 2016MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videoscited by 193
  21. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 188
  22. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 183
  23. 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 177
  24. 2018Learning Factorized Multimodal Representationscited by 171
  25. 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 159
  26. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  27. 2014Cross-view Action Modeling, Learning and Recognitioncited by 134
  28. 2023Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentationcited by 128
  29. 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
  30. 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 115
  31. 2019MMKG: Multi-Modal Knowledge Graphscited by 104
  32. 2018Learning a Text-Video Embedding from Incomplete and Heterogeneous Datacited by 103
  33. 2018Multi-attention Recurrent Network for Human Communication Comprehensioncited by 89
  34. 2018CentralNet: a Multilayer Approach for Multimodal Fusioncited by 86
  35. 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 86
  36. 2023OtterHD: A High-Resolution Multi-modality Modelcited by 72
  37. 2022M3Care: Learning with Missing Modalities in Multimodal Healthcare Datacited by 71
  38. 2021Knowledge distillation from multi-modal to mono-modal segmentation networkscited by 68

50 most cited of 119 All 119 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 27; 2026: 61.

89 papers.

Sort:

By area:

  1. 2025SimMLM: A Simple Framework for Multi-modal Learning with Missing Modalitycited by 22
  2. 2024Towards Modality Generalization: A Benchmark and Prospective Analysiscited by 10
  3. 2025Multimodal Federated Learning With Missing Modalities through Feature Imputation Networkcited by 8
  4. 2025Synergistic Prompting for Robust Visual Recognition with Missing Modalitiescited by 8
  5. 2026Omni-Modal Dissonance Benchmark: Systematically Breaking Modality Consensus to Probe Robustness and Calibrated Abstentioncited by 6
  6. 2025UniMoCo: Unified Modality Completion for Robust Multi-Modal Embeddingscited by 5
  7. 2025Cohort-Based Active Modality Acquisitioncited by 4
  8. 2025Fusion or Confusion? Multimodal Complexity Is Not All You Needcited by 4
  9. 2026CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learningcited by 4
  10. 2025Multimodal Representation Learning and Fusioncited by 3
  11. 2025Calibrated Multimodal Representation Learning with Missing Modalitiescited by 3
  12. 2026Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classificationcited by 3
38 more
  1. 2025ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysiscited by 2
  2. 2025Towards Robust Multimodal Representation: A Unified Approach with Adaptive Experts and Alignmentcited by 2
  3. 2025Towards deployment-centric multimodal AI beyond vision and languagecited by 2
  4. 2025Not Only Grey Matter: OmniBrain for Robust Multimodal Classification of Alzheimer's Diseasecited by 2
  5. 2025Multi-modal Co-learning for Earth Observation: Enhancing single-modality models via modality collaborationcited by 2
  6. 2025Multi-Modal Masked Autoencoders for Learning Image-Spectrum Associations for Galaxy Evolution and Cosmologycited by 2
  7. 2026A Multimodal Dataset of Student Oral Presentations with Sensors and Evaluation Datacited by 2
  8. 2026MetaPerch: Learning from metadata for bioacoustics foundation modelscited by 2
  9. 2025ICYM2I: The illusion of multimodal informativeness under missingnesscited by 1
  10. 2025Multimodal Foundation Models for Early Disease Detectioncited by 1
  11. 2025PhysioME: A Robust Multimodal Self-Supervised Framework for Physiological Signals with Missing Modalitiescited by 1
  12. 2025OmniField: Conditioned Neural Fields for Robust Multimodal Spatiotemporal Learningcited by 1
  13. 2026PyHealth 2.0: A Comprehensive Open-Source Toolkit for Accessible and Reproducible Clinical Deep Learningcited by 1
  14. 2026Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflectioncited by 1
  15. 2026ModalImmune: Immunity Driven Unlearning via Self Destructive Trainingcited by 1
  16. 2026REMIND: Rethinking Medical High-Modality Learning under Missingness--A Long-Tailed Distribution Perspectivecited by 1
  17. 2026BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Ratescited by 1
  18. 2026Memory Bear AI Memory Science Engine for Multimodal Affective Intelligence: A Technical Reportcited by 1
  19. 2026Evaluation Before Generation: A Paradigm for Robust Multimodal Sentiment Analysis with Missing Modalitiescited by 1
  20. 2026Retrieving to Recover: Towards Incomplete Audio-Visual Question Answering via Semantic-consistent Purificationcited by 1
  21. 2026Multi-Perspective Evidence Synthesis and Reasoning for Unsupervised Multimodal Entity Linkingcited by 1
  22. 2026LIMSSR: LLM-Driven Sequence-to-Score Reasoning under Training-Time Incomplete Multimodal Observationscited by 1
  23. 2026Controlling Decision Drift in Multimodal Sentiment Analysis with Missing Modalitiescited by 1
  24. 2026Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphscited by 1
  25. 2026Conformal Calibration for Multi-Modal Regression with Missing Modalitiescited by 1
  26. 2025Multimodal Knowledge Distillation for Egocentric Action Recognition Robust to Missing Modalitiescited by 0
  27. 2025Synthetic Perception: Can Generated Images Unlock Latent Visual Prior for Text-Centric Reasoning?cited by 0
  28. 2025Robust Incomplete-Modality Alignment for Ophthalmic Disease Grading and Diagnosis via Labeled Optimal Transportcited by 0
  29. 2025Multimodal Sheaf-based Network for Glioblastoma Molecular Subtype Predictioncited by 0
  30. 2025RLBind: Adversarial-Invariant Cross-Modal Alignment for Unified Robust Embeddingscited by 0
  31. 2025Lyapunov-Stable Adaptive Control for Multimodal Concept Driftcited by 0
  32. 2025MedM2T: A MultiModal Framework for Time-Aware Modeling with Electronic Health Record and Electrocardiogram Datacited by 0
  33. 2025RobustA: Robust Anomaly Detection in Multimodal Datacited by 0
  34. 2025Interpretable Alzheimer's Diagnosis via Multimodal Fusion of Regional Brain Expertscited by 0
  35. 2026AviationLMM: A Large Multimodal Foundation Model for Civil Aviationcited by 0
  36. 2026DIS2: Disentanglement Meets Distillation with Classwise Attention for Robust Remote Sensing Segmentation under Missing Modalitiescited by 0
  37. 2026Bayesian Integration of Nonlinear Incomplete Clinical Datacited by 0
  38. 2026OMG-Agent: Toward Robust Missing Modality Generation with Decoupled Coarse-to-Fine Agentic Workflowscited by 0

50 most cited of 89 All 89 (JSON)