# Mean-Field Approximation to Gaussian-Softmax Integral with Application to Uncertainty Estimation

arXiv: https://arxiv.org/abs/2006.07584 · HTML: https://mlsanity.com/paper/2006.07584/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2020.

## Built on by (most cited first, 16 of 16)

- [A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness](https://mlsanity.com/paper/2205.00403/) (2022)
- [Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning](https://mlsanity.com/paper/2111.03577/) (2021)
- [Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networks](https://mlsanity.com/paper/2205.10041/) (2022)
- [Massively Scaling Heteroscedastic Classifiers](https://mlsanity.com/paper/2301.12860/) (2023)
- [Kalman Bayesian Neural Networks for Closed-form Online Learning](https://mlsanity.com/paper/2110.00944/) (2021)
- [laplax -- Laplace Approximations with JAX](https://mlsanity.com/paper/2507.17013/) (2025)
- [Bayesian Mixture of Experts For Large Language Models](https://mlsanity.com/paper/2511.08968/) (2025)
- [Efficient Bayesian Updates for Deep Active Learning via Laplace Approximations](https://mlsanity.com/paper/2210.06112/) (2022)
- [Efficient Uncertainty Estimation with Gaussian Process for Reliable Dialog Response Retrieval](https://mlsanity.com/paper/2303.08599/) (2023)
- [Federated Learning with Uncertainty-Based Client Clustering for Fleet-Wide Fault Diagnosis](https://mlsanity.com/paper/2304.13275/) (2023)
- [Transferable Candidate Proposal with Bounded Uncertainty](https://mlsanity.com/paper/2312.04604/) (2023)
- [Rethinking Approximate Gaussian Inference in Classification](https://mlsanity.com/paper/2502.03366/) (2025)
- [Adaptive Catalyst Discovery Using Multicriteria Bayesian Optimization with Representation Learning](https://mlsanity.com/paper/2404.12445/) (2024)
- [Variational Bayesian Pseudo-Coreset](https://mlsanity.com/paper/2502.21143/) (2025)
- [Calibrated Sampling-Free Uncertainty Estimation in Bayesian Deep Learning](https://mlsanity.com/paper/2606.16214/) (2026)
- [Q-based Variational Inverse Reinforcement Learning](https://mlsanity.com/paper/2608.16888/) (2026)

## Abstract (arXiv)

Many methods have been proposed to quantify the predictive uncertainty associated with the outputs of deep neural networks. Among them, ensemble methods often lead to state-of-the-art results, though they require modifications to the training procedures and are computationally costly for both training and inference. In this paper, we propose a new single-model based approach. The main idea is inspired by the observation that we can "simulate" an ensemble of models by drawing from a Gaussian distribution, with a form similar to those from the asymptotic normality theory, infinitesimal Jackknife, Laplacian approximation to Bayesian neural networks, and trajectories in stochastic gradient descents. However, instead of using each model in the "ensemble" to predict and then aggregating their predictions, we integrate the Gaussian distribution and the softmax outputs of the neural networks. We use a mean-field approximation formula to compute this analytically intractable integral. The proposed approach has several appealing properties: it functions as an ensemble without requiring multiple models, and it enables closed-form approximate inference using only the first and second moments of the Gaussian. Empirically, the proposed approach performs competitively when compared to state-of-the-art methods, including deep ensembles, temperature scaling, dropout and Bayesian NNs, on standard uncertainty estimation tasks. It also outperforms many methods on out-of-distribution detection.