mlsanityData to 14 Oct 2026

Provable Dynamic Fusion for Low-Quality Multimodal Data

arXiv 2306.02050 ↗First posted 3 Jun 2023cs.LG cs.CVcited by 82

Not on the map. Why?

Abstract (arXiv)

The inherent challenge of multimodal fusion is to precisely capture the cross-modal correlation and flexibly conduct cross-modal interaction. To fully release the value of each modality and mitigate the influence of low-quality multimodal data, dynamic multimodal fusion emerges as a promising learning paradigm. Despite its widespread use, theoretical justifications in this field are still notably lacking. Can we design a provably robust multimodal fusion method? This paper provides theoretical understandings to answer this question under a most popular multimodal fusion framework from the generalization perspective. We proceed to reveal that several uncertainty estimation solutions are naturally available to achieve robust multimodal fusion. Then a novel multimodal fusion framework termed Quality-aware Multimodal Fusion (QMF) is proposed, which can improve the performance in terms of classification accuracy and model robustness. Extensive experimental results on multiple benchmarks can support our findings.

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). Builds on 57 of its 99 references that are in the map. First posted on arXiv 3 Jun 2023.

Builds on

57 of 99 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  6. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  7. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  8. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  9. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 705
  10. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  11. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  12. 2019Measuring Calibration in Deep Learningcited by 446
38 more
  1. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 432
  2. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
  3. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 328
  4. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 264
  5. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 215
  6. 2020Training independent subnetworks for robust predictioncited by 177
  7. 2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 168
  8. 2020Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Countscited by 164
  9. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 153
  10. 2022Multimodal Token Fusion for Vision Transformerscited by 136
  11. 2019Supervised Multimodal Bitransformers for Classifying Images and Textcited by 130
  12. 2019MMTM: Multimodal Transfer Module for CNN Fusioncited by 129
  13. 2022Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)cited by 122
  14. 2020Deep Multimodal Fusion by Channel Exchangingcited by 118
  15. 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
  16. 2018Bias-Reduced Uncertainty Estimation for Deep Neural Classifierscited by 113
  17. 2020Confidence-Aware Learning for Deep Neural Networkscited by 113
  18. 2019Multimodal End-to-End Autonomous Drivingcited by 110
  19. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  20. 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
  21. 2022Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networkscited by 95
  22. 2021Trusted Multi-View Classificationcited by 94
  23. 2022Training OOD Detectors in their Natural Habitatscited by 86
  24. 2020Efficient RGB-D Semantic Segmentation for Indoor Scene Analysiscited by 84
  25. 2020Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problemscited by 83
  26. 2019Calibration tests in multi-class classification: A unifying frameworkcited by 79
  27. 2017Data Noising as Smoothing in Neural Network Language Modelscited by 78
  28. 2020Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learningcited by 74
  29. 2019Towards neural networks that provably know when they don't knowcited by 69
  30. 2021Multimodal Object Detection via Probabilistic Ensemblingcited by 68
  31. 2015Multimodal Deep Learning for Robust RGB-D Object Recognitioncited by 57
  32. 2018Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detectioncited by 57
  33. 2019Learning Data Manipulation for Augmentation and Weightingcited by 45
  34. 2021Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributionscited by 39
  35. 2015Deep Multimodal Learning for Audio-Visual Speech Recognitioncited by 38
  36. 2022UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixupcited by 34
  37. 2022Efficient Multi-Task RGB-D Scene Analysis for Indoor Environmentscited by 33
  38. 2022Uncertainty-Aware Multi-View Representation Learningcited by 29

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 24; 2025: 26; 2026: 28.

82 papers.

Sort:

By area:

  1. 2023Multimodal Representation Learning by Alternating Unimodal Adaptationcited by 75
  2. 2024ReconBoost: Boosting Can Achieve Modality Reconcilementcited by 60
  3. 2024Task-Customized Mixture of Adapters for General Image Fusioncited by 59
  4. 2024Multimodal Fusion on Low-quality Data: A Comprehensive Surveycited by 49
  5. 2024Reliable Conflictive Multi-View Learningcited by 45
  6. 2023ID-like Prompt Learning for Few-Shot Out-of-Distribution Detectioncited by 42
  7. 2024Classifier-guided Gradient Modulation for Enhanced Multimodal Learningcited by 35
  8. 2024COME: Test-time adaption by Conservatively Minimizing Entropycited by 31
  9. 2025Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Priorcited by 29
  10. 2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 24
  11. 2024Learning on Multimodal Graphs: A Surveycited by 22
  12. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
38 more
  1. 2024Play to the Score: Stage-Guided Dynamic Multi-Sensory Fusion for Robotic Manipulationcited by 20
  2. 2024CRSOT: Cross-Resolution Object Tracking using Unaligned Frame and Event Camerascited by 17
  3. 2024Predictive Dynamic Fusioncited by 17
  4. 2024Suppress and Rebalance: Towards Generalized Multi-Modal Face Anti-Spoofingcited by 16
  5. 2025DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learningcited by 16
  6. 2024A Survey on Safe Multi-Modal Learning Systemcited by 15
  7. 2024DirMixE: Harnessing Test Agnostic Long-tail Recognition with Hierarchical Label Variationscited by 15
  8. 2025Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoringcited by 10
  9. 2025Anomaly Detection and Generation with Diffusion Models: A Surveycited by 8
  10. 2026Unbiased Dynamic Multimodal Fusioncited by 8
  11. 2024Test-Time Dynamic Image Fusioncited by 6
  12. 2025Cooperation of Experts: Fusing Heterogeneous Information with Large Margincited by 6
  13. 2025Fusion Meets Diverse Conditions: A High-diversity Benchmark and Baseline for UAV-based Multimodal Object Detection with Condition Cuescited by 6
  14. 2023Noisy Correspondence Learning with Self-Reinforcing Errors Mitigationcited by 5
  15. 2024Credibility-Aware Multi-Modal Fusion Using Probabilistic Circuitscited by 5
  16. 2024Confidence-aware multi-modality learning for eye disease screeningcited by 5
  17. 2024Towards the Causal Complete Cause of Multi-Modal Representation Learningcited by 5
  18. 2025G2D: Boosting Multimodal Learning with Gradient-Guided Distillationcited by 5
  19. 2026CLCR: Cross-Level Semantic Collaborative Representation for Multimodal Learningcited by 5
  20. 2024Let's Fuse Step by Step: A Generative Fusion Decoding Algorithm with LLMs for Robust and Instruction-Aware ASR and OCRcited by 4
  21. 2026Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classificationcited by 4
  22. 2025Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbationcited by 3
  23. 2025Multimodal Negative Learningcited by 3
  24. 2026Multimodal Deep Generative Model for Semi-Supervised Learning under Class Imbalancecited by 3
  25. 2024AdaFlow: Opportunistic Inference on Asynchronous Mobile Data with Generalized Affinity Controlcited by 2
  26. 2025Robust Multimodal Learning via Cross-Modal Proxy Tokenscited by 2
  27. 2025Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysiscited by 2
  28. 2025Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learningcited by 2
  29. 2026Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learningcited by 2
  30. 2026Multimodal Classification via Total Correlation Maximizationcited by 2
  31. 2026Context-specific Credibility-aware Multimodal Fusion with Conditional Probabilistic Circuitscited by 2
  32. 2024BIG-MoE: Bypass Isolated Gating MoE for Generalized Multimodal Face Anti-Spoofingcited by 1
  33. 2025Reliable Disentanglement Multi-view Learning Against View Adversarial Attackscited by 1
  34. 2025RollingQ: Reviving the Cooperation Dynamics in Multimodal Transformercited by 1
  35. 2025Multi-modal Uncertainty Robust Tree Cover Segmentation For High-Resolution Remote Sensing Imagescited by 1
  36. 2025MULTIBENCH++: A Unified and Comprehensive Multimodal Fusion Benchmarking Across Specialized Domainscited by 1
  37. 2025Robust Multimodal Sentiment Analysis of Image-Text Pairs by Distribution-Based Feature Recovery and Fusioncited by 1
  38. 2026M3D-BFS: a Multi-stage Dynamic Fusion Strategy for Sample-Adaptive Multi-Modal Brain Network Analysiscited by 1

50 most cited of 82 All 82 (JSON)