# Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions

arXiv: https://arxiv.org/abs/2111.08456 · HTML: https://mlsanity.com/paper/2111.08456/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 57 references that are in the map.
- First posted on arXiv 11 Nov 2021.

## Builds on (most cited first, 28 of 28)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Calibrating Deep Neural Networks using Focal Loss](https://mlsanity.com/paper/2002.09437/) (2020)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Variational Dropout Sparsifies Deep Neural Networks](https://mlsanity.com/paper/1701.05369/) (2017)
- [MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos](https://mlsanity.com/paper/1606.06259/) (2016)
- [Addressing Failure Prediction by Learning Model Confidence](https://mlsanity.com/paper/1910.04851/) (2019)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities](https://mlsanity.com/paper/1812.07809/) (2018)
- [Evaluating Bayesian Deep Learning Methods for Semantic Segmentation](https://mlsanity.com/paper/1811.12709/) (2018)
- [Trusted Multi-View Classification](https://mlsanity.com/paper/2102.02051/) (2021)
- [Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors](https://mlsanity.com/paper/1811.09362/) (2018)
- [Deep Partial Multi-View Learning](https://mlsanity.com/paper/2011.06170/) (2020)
- [3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training](https://mlsanity.com/paper/1811.12506/) (2018)
- [Efficient Large-Scale Multi-Modal Classification](https://mlsanity.com/paper/1802.02892/) (2018)
- [Depth Uncertainty in Neural Networks](https://mlsanity.com/paper/2006.08437/) (2020)
- [Multimodal Language Analysis with Recurrent Multistage Fusion](https://mlsanity.com/paper/1808.03920/) (2018)
- [Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization](https://mlsanity.com/paper/1907.01011/) (2019)
- [Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detection](https://mlsanity.com/paper/2012.05329/) (2020)
- [Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors](https://mlsanity.com/paper/2012.00201/) (2020)
- [Shared Representation Learning for Heterogeneous Face Recognition](https://mlsanity.com/paper/1406.1247/) (2014)

## Built on by (most cited first, 30 of 39)

- [Provable Dynamic Fusion for Low-Quality Multimodal Data](https://mlsanity.com/paper/2306.02050/) (2023)
- [A Review of Uncertainty Estimation and its Application in Medical Imaging](https://mlsanity.com/paper/2302.08119/) (2023)
- [Multimodal Fusion on Low-quality Data: A Comprehensive Survey](https://mlsanity.com/paper/2404.18947/) (2024)
- [On Second-Order Scoring Rules for Epistemic Uncertainty Quantification](https://mlsanity.com/paper/2301.12736/) (2023)
- [Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?](https://mlsanity.com/paper/2402.09056/) (2024)
- [UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixup](https://mlsanity.com/paper/2209.08928/) (2022)
- [ELFNet: Evidential Local-global Fusion for Stereo Matching](https://mlsanity.com/paper/2308.00728/) (2023)
- [A Comprehensive Survey on Evidential Deep Learning and Its Applications](https://mlsanity.com/paper/2409.04720/) (2024)
- [Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage?](https://mlsanity.com/paper/2402.06160/) (2024)
- [Predictive Dynamic Fusion](https://mlsanity.com/paper/2406.04802/) (2024)
- [Learning to Rebalance Multi-Modal Optimization by Adaptively Masking Subnetworks](https://mlsanity.com/paper/2404.08347/) (2024)
- [Revisiting Essential and Nonessential Settings of Evidential Deep Learning](https://mlsanity.com/paper/2410.00393/) (2024)
- [Beyond Unimodal: Generalising Neural Processes for Multimodal Uncertainty Estimation](https://mlsanity.com/paper/2304.01518/) (2023)
- [Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture](https://mlsanity.com/paper/2210.02676/) (2022)
- [Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring](https://mlsanity.com/paper/2508.09085/) (2025)
- [Trustworthy Multimodal Fusion for Sentiment Analysis in Ordinal Sentiment Space](https://mlsanity.com/paper/2404.08923/) (2024)
- [Towards the Causal Complete Cause of Multi-Modal Representation Learning](https://mlsanity.com/paper/2407.14058/) (2024)
- [dugMatting: Decomposed-Uncertainty-Guided Matting](https://mlsanity.com/paper/2306.01452/) (2023)
- [Evidential Neural Radiance Fields](https://mlsanity.com/paper/2602.23574/) (2026)
- [Reweighted Mixup for Subpopulation Shift](https://mlsanity.com/paper/2304.04148/) (2023)
- [Multi-modal MRI Translation via Evidential Regression and Distribution Calibration](https://mlsanity.com/paper/2407.07372/) (2024)
- [Explainable Multimodal Regression via Information Decomposition](https://mlsanity.com/paper/2512.22102/) (2025)
- [Context-specific Credibility-aware Multimodal Fusion with Conditional Probabilistic Circuits](https://mlsanity.com/paper/2603.26629/) (2026)
- [Improving Uncertainty Quantification of Variance Networks by Tree-Structured Learning](https://mlsanity.com/paper/2212.12658/) (2022)
- [Cross-Dataset Gaze Estimation by Evidential Inter-intra Fusion](https://mlsanity.com/paper/2409.04766/) (2024)
- [Robust Multimodal Sentiment Analysis of Image-Text Pairs by Distribution-Based Feature Recovery and Fusion](https://mlsanity.com/paper/2511.18751/) (2025)
- [Gaussian Mixture based Evidential Learning for Stereo Matching](https://mlsanity.com/paper/2408.02796/) (2024)
- [Multi-QuAD: Multi-Level Quality-Adaptive Dynamic Network for Reliable Multimodal Classification](https://mlsanity.com/paper/2412.14489/) (2024)
- [CEC-MMR: Cross-Entropy Clustering Approach to Multi-Modal Regression](https://mlsanity.com/paper/2504.07301/) (2025)
- [UGPL: Uncertainty-Guided Progressive Learning for Evidence-Based Classification in Computed Tomography](https://mlsanity.com/paper/2507.14102/) (2025)

## Abstract (arXiv)

Multimodal regression is a fundamental task, which integrates the information from different sources to improve the performance of follow-up applications. However, existing methods mainly focus on improving the performance and often ignore the confidence of prediction for diverse situations. In this study, we are devoted to trustworthy multimodal regression which is critical in cost-sensitive domains. To this end, we introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Our model can be dynamically aware of uncertainty for each modality, and also robust for corrupted modalities. Furthermore, the proposed MoNIG ensures explicitly representation of (modality-specific/global) epistemic and aleatoric uncertainties, respectively. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks (e.g., temperature prediction for superconductivity, relative location prediction for CT slices, and multimodal sentiment analysis).