mlsanityData to 14 Oct 2026

General Incomplete Multimodal Learning via Dynamic Quality Perception

arXiv 2607.06943 ↗First posted 8 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, where entire modalities are absent, while overlooking intra-modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete Multimodal Learning (GIML), a unified framework that simultaneously handles both inter-modality missing and intra-modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation-aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise-aware Quality Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise-Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: https://github.com/Yu-Five/GIML.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 60 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

44 of 60 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  4. 2017Look, Listen and Learncited by 458
  5. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  6. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
  7. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 264
  8. 2019Probabilistic Face Embeddingscited by 173
  9. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
  10. 2016HeMIS: Hetero-Modal Image Segmentationcited by 103
  11. 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 95
  12. 2018Multi-attention Recurrent Network for Human Communication Comprehensioncited by 89
32 more
  1. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 82
  2. 2024Variational Bayesian Last Layerscited by 69
  3. 2024FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusioncited by 68
  4. 2018D3D: Distilled 3D Networks for Video Action Recognitioncited by 60
  5. 2023Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusioncited by 54
  6. 2023Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modalitycited by 45
  7. 2023MMANet: Margin-aware Distillation and Modality-aware Regularization for Incomplete Multimodal Learningcited by 40
  8. 2024CrossFuse: A Novel Cross Attention Mechanism based Infrared and Visible Image Fusion Approachcited by 38
  9. 2025CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answeringcited by 37
  10. 2025A Closer Look at Multimodal Representation Collapsecited by 28
  11. 2024Towards Robust Multimodal Sentiment Analysis with Incomplete Datacited by 27
  12. 2025SimMLM: A Simple Framework for Multi-modal Learning with Missing Modalitycited by 23
  13. 2025Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformerscited by 20
  14. 2024Robust Multimodal Learning via Representation Decouplingcited by 17
  15. 2025Improving Multimodal Learning via Imbalanced Learningcited by 17
  16. 2025CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignmentcited by 16
  17. 2025Boosting Multimodal Learning via Disentangled Gradient Learningcited by 16
  18. 2023RedCore: Relative Advantage Aware Cross-modal Representation Learning for Missing Modalities with Imbalanced Missing Ratescited by 15
  19. 2024Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognitioncited by 15
  20. 2019Integrating cross-modality hallucinated MRI with CT to aid mediastinal lung tumor segmentationcited by 14
  21. 2025Incomplete Multi-view Clustering via Diffusion Contrastive Generationcited by 13
  22. 2024A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognitioncited by 12
  23. 2025Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable Modelscited by 9
  24. 2024Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptionscited by 8
  25. 2026Unbiased Dynamic Multimodal Fusioncited by 8
  26. 2024Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimationcited by 7
  27. 2025EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Modelscited by 5
  28. 2025RoHyDR: Robust Hybrid Diffusion Recovery for Incomplete Multimodal Emotion Recognitioncited by 3
  29. 2025Beyond Simple Fusion: Adaptive Gated Fusion for Robust Multimodal Sentiment Analysiscited by 3
  30. 2025TMDC: A Two-Stage Modality Denoising and Complementation Framework for Multimodal Sentiment Analysis with Missing and Noisy Modalitiescited by 3
  31. 2025Intra-Class Probabilistic Embeddings for Uncertainty Estimation in Vision-Language Modelscited by 2
  32. 2025Unreliable Uncertainty Estimates with Monte Carlo Dropoutcited by 2

Built on by

Not cited yet.