mlsanityData to 7 Oct 2026

Predictive Dynamic Fusion

arXiv 2406.04802 ↗First posted 7 Jun 2024cs.CV cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Multimodal fusion is crucial in joint decision-making systems for rendering holistic judgments. Since multimodal data changes in open environments, dynamic fusion has emerged and achieved remarkable progress in numerous applications. However, most existing dynamic multimodal fusion methods lack theoretical guarantees and easily fall into suboptimal problems, yielding unreliability and instability. To address this issue, we propose a Predictive Dynamic Fusion (PDF) framework for multimodal learning. We proceed to reveal the multimodal fusion from a generalization perspective and theoretically derive the predictable Collaborative Belief (Co-Belief) with Mono- and Holo-Confidence, which provably reduces the upper bound of generalization error. Accordingly, we further propose a relative calibration strategy to calibrate the predicted Co-Belief for potential uncertainty. Extensive experiments on multiple benchmarks confirm our superiority. Our code is available at https://github.com/Yinan-Xia/PDF.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 30 of its 58 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

30 of 58 references.

References · 30

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  4. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  5. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  6. 2019When Does Label Smoothing Help?cited by 985
  7. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  8. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  9. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  10. 2019Deep Evidential Regressioncited by 354
  11. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 340
  12. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
18 more
  1. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 313
  2. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  3. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
  4. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  5. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  6. 2019Addressing Failure Prediction by Learning Model Confidencecited by 183
  7. 2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 167
  8. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 151
  9. 2019Supervised Multimodal Bitransformers for Classifying Images and Textcited by 130
  10. 2019MFAS: Multimodal Fusion Architecture Searchcited by 84
  11. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 81
  12. 2017Data Noising as Smoothing in Neural Network Language Modelscited by 78
  13. 2024Task-Customized Mixture of Adapters for General Image Fusioncited by 59
  14. 2020Gradients as a Measure of Uncertainty in Neural Networkscited by 55
  15. 2022Dynamic Multimodal Fusioncited by 54
  16. 2023Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusioncited by 54
  17. 2021Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributionscited by 39
  18. 2023Calibrating Multimodal Learningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 7.

17 papers.

Sort:

  1. 2025DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learningcited by 16
  2. 2025Multimodal Cancer Survival Analysis via Hypergraph Learning with Cross-Modality Rebalancecited by 11
  3. 2024MM-GTUNets: Unified Multi-Modal Graph Deep Learning for Brain Disorders Predictioncited by 9
  4. 2025Fusion or Confusion? Multimodal Complexity Is Not All You Needcited by 4
  5. 2026Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classificationcited by 3
  6. 2026Multimodal Deep Generative Model for Semi-Supervised Learning under Class Imbalancecited by 3
  7. 2025Integrating Biological and Machine Intelligence: Attention Mechanisms in Brain-Computer Interfacescited by 2
  8. 2025Multimodal Negative Learningcited by 2
  9. 2025Reliable Disentanglement Multi-view Learning Against View Adversarial Attackscited by 1
  10. 2024Multi-QuAD: Multi-Level Quality-Adaptive Dynamic Network for Reliable Multimodal Classificationcited by 0
  11. 2025Reliable Multimodal Learning Via Multi-Level Adaptive DeConfusioncited by 0
  12. 2025AIM: Adaptive Intra-Network Modulation for Balanced Multimodal Learningcited by 0
5 more
  1. 2026Test-time Adaptive Hierarchical Co-enhanced Denoising Network for Reliable Multimodal Classificationcited by 0
  2. 2026Multi-modal Test-time Adaptation via Adaptive Probabilistic Gaussian Calibrationcited by 0
  3. 2026Robust Fuzzy Multi-view Learning under View Conflictcited by 0
  4. 2026Geometry-based Schr"odinger Bridges for Trustworthy Multimodal Fusioncited by 0
  5. 2026Quality-Aware Multimodal Fusion Reveals Implicit Identity in Valence-Arousal Featurescited by 0