# Inferential Moments of Uncertain Multivariable Systems

arXiv: https://arxiv.org/abs/2305.01841 · HTML: https://mlsanity.com/paper/2305.01841/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 52 references that are in the map.
- First posted on arXiv 3 May 2023.

## Builds on (most cited first, 8 of 8)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [A survey of Bayesian Network structure learning](https://mlsanity.com/paper/2109.11415/) (2021)
- [A General Model Validation and Testing Tool](https://mlsanity.com/paper/1908.11251/) (2019)
- [Entropic Causal Inference for Neurological Applications](https://mlsanity.com/paper/2010.04247/) (2020)
- [Interaction Networks from Discrete Event Data by Poisson Multivariate Mutual Information Estimation and Information Flow with Applications from Gene Expression Data](https://mlsanity.com/paper/2002.02477/) (2020)
- [The Design of Global Correlation Quantifiers and Continuous Notions of Statistical Sufficiency](https://mlsanity.com/paper/1907.06992/) (2019)

## Abstract (arXiv)

This article expands the framework of Bayesian inference and provides direct probabilistic methods for approaching inference tasks that are typically handled with information theory. We treat Bayesian probability updating as a random process and uncover intrinsic quantitative features of joint probability distributions called inferential moments. Inferential moments quantify shape information about how a prior distribution is expected to update in response to yet to be obtained information. Further, we quantify the unique probability distribution whose statistical moments are the inferential moments in question. We find a power series expansion of the mutual information in terms of inferential moments, which implies a connection between inferential theoretic logic and elements of information theory. Of particular interest is the inferential deviation, which is the expected variation of the probability of one variable in response to an inferential update of another. We explore two applications that analyze the inferential deviations of a Bayesian network to improve decision-making. We implement simple greedy algorithms for exploring sensor tasking using inferential deviations that generally outperform similar greedy mutual information algorithms in terms of root mean squared error between epistemic probability estimates and the ground truth probabilities they are estimating.