# Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisition

arXiv: https://arxiv.org/abs/2501.18268 · HTML: https://mlsanity.com/paper/2501.18268/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 88 references that are in the map.
- First posted on arXiv 30 Jan 2025.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Out-of-Distribution Detection with Deep Nearest Neighbors](https://mlsanity.com/paper/2204.06507/) (2022)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts](https://mlsanity.com/paper/2006.09239/) (2020)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?](https://mlsanity.com/paper/2209.03302/) (2022)
- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [Class-Conditional Conformal Prediction with Many Classes](https://mlsanity.com/paper/2306.09335/) (2023)
- [Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness](https://mlsanity.com/paper/1905.13472/) (2019)
- [DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks](https://mlsanity.com/paper/1906.04569/) (2019)
- [Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks](https://mlsanity.com/paper/2402.19460/) (2024)
- [Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation](https://mlsanity.com/paper/2110.03051/) (2021)
- [Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation](https://mlsanity.com/paper/2203.06102/) (2022)
- [Active Feature Acquisition with Generative Surrogate Models](https://mlsanity.com/paper/2010.02433/) (2020)
- [Aleatoric and Epistemic Uncertainty with Random Forests](https://mlsanity.com/paper/2001.00893/) (2020)
- [The Unreasonable Effectiveness of Deep Evidential Regression](https://mlsanity.com/paper/2205.10060/) (2022)
- [Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates](https://mlsanity.com/paper/2101.08133/) (2021)
- [On Second-Order Scoring Rules for Epistemic Uncertainty Quantification](https://mlsanity.com/paper/2301.12736/) (2023)
- [Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?](https://mlsanity.com/paper/2402.09056/) (2024)
- [Introducing an Improved Information-Theoretic Measure of Predictive Uncertainty](https://mlsanity.com/paper/2311.08309/) (2023)
- [BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity](https://mlsanity.com/paper/2406.12723/) (2024)

## Built on by (most cited first, 5 of 5)

- [Cohort-Based Active Modality Acquisition](https://mlsanity.com/paper/2505.16791/) (2025)
- [UbiQTree: Uncertainty Quantification in XAI with Tree Ensembles](https://mlsanity.com/paper/2508.09639/) (2025)
- [Robust Explanations Through Uncertainty Decomposition: A Path to Trustworthier AI](https://mlsanity.com/paper/2507.12913/) (2025)
- [Conditional Validity for Adaptive Modality Acquisition: When the Policy Chooses Its Own Calibration Group](https://mlsanity.com/paper/2608.15520/) (2026)
- [Explanations-Driven Active Feature Acquisition for Algorithmic Recourse](https://mlsanity.com/paper/2609.12179/) (2026)

## Abstract (arXiv)

To generate accurate and reliable predictions, modern AI systems need to combine data from multiple modalities, such as text, images, audio, spreadsheets, and time series. Multi-modal data introduces new opportunities and challenges for disentangling uncertainty: it is commonly assumed in the machine learning community that epistemic uncertainty can be reduced by collecting more data, while aleatoric uncertainty is irreducible. However, this assumption is challenged in modern AI systems when information is obtained from different modalities. This paper introduces an innovative data acquisition framework where uncertainty disentanglement leads to actionable decisions, allowing sampling in two directions: sample size and data modality. The main hypothesis is that aleatoric uncertainty decreases as the number of modalities increases, while epistemic uncertainty decreases by collecting more observations. We provide proof-of-concept implementations on two multi-modal datasets to showcase our data acquisition framework, which combines ideas from active learning, active feature acquisition and uncertainty quantification.