# Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions

arXiv: https://arxiv.org/abs/2105.04471 · HTML: https://mlsanity.com/paper/2105.04471/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 95 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 30 of 57)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning](https://mlsanity.com/paper/2002.06715/) (2020)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Understanding Deep Neural Networks with Rectified Linear Units](https://mlsanity.com/paper/1611.01491/) (2016)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Why Normalizing Flows Fail to Detect Out-of-Distribution Data](https://mlsanity.com/paper/2006.08545/) (2020)
- [Contrastive Training for Improved Out-of-Distribution Detection](https://mlsanity.com/paper/2007.05566/) (2020)

## Built on by (most cited first, 13 of 13)

- [Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Models](https://mlsanity.com/paper/2406.18580/) (2024)
- [Uncertainty Estimation by Density Aware Evidential Deep Learning](https://mlsanity.com/paper/2409.08754/) (2024)
- [Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage?](https://mlsanity.com/paper/2402.06160/) (2024)
- [Revisiting Essential and Nonessential Settings of Evidential Deep Learning](https://mlsanity.com/paper/2410.00393/) (2024)
- [Uncertainty Estimation for Molecules: Desiderata and Methods](https://mlsanity.com/paper/2306.14916/) (2023)
- [Position: Epistemic uncertainty estimation methods are fundamentally incomplete](https://mlsanity.com/paper/2505.23506/) (2025)
- [Topology-Matching Normalizing Flows for Out-of-Distribution Detection in Robot Learning](https://mlsanity.com/paper/2311.06481/) (2023)
- [A Unified Approach Towards Active Learning and Out-of-Distribution Detection](https://mlsanity.com/paper/2405.11337/) (2024)
- [Joint Out-of-Distribution Filtering and Data Discovery Active Learning](https://mlsanity.com/paper/2503.02491/) (2025)
- [Prior2Former -- Evidential Modeling of Mask Transformers for Assumption-Free Open-World Panoptic Segmentation](https://mlsanity.com/paper/2504.04841/) (2025)
- [Discriminant Distance-Aware Representation on Deterministic Uncertainty Quantification Methods](https://mlsanity.com/paper/2402.12664/) (2024)
- [PRESCRIBE: Predicting Single-Cell Responses with Bayesian Estimation](https://mlsanity.com/paper/2510.07964/) (2025)
- [Efficient Epistemic Uncertainty Estimation in Regression Ensemble Models Using Pairwise-Distance Estimators](https://mlsanity.com/paper/2308.13498/) (2023)

## Abstract (arXiv)

Uncertainty awareness is crucial to develop reliable machine learning models. In this work, we propose the Natural Posterior Network (NatPN) for fast and high-quality uncertainty estimation for any task where the target distribution belongs to the exponential family. Thus, NatPN finds application for both classification and general regression settings. Unlike many previous approaches, NatPN does not require out-of-distribution (OOD) data at training time. Instead, it leverages Normalizing Flows to fit a single density on a learned low-dimensional and task-dependent latent space. For any input sample, NatPN uses the predicted likelihood to perform a Bayesian update over the target distribution. Theoretically, NatPN assigns high uncertainty far away from training data. Empirically, our extensive experiments on calibration and OOD detection show that NatPN delivers highly competitive performance for classification, regression and count prediction tasks.