mlsanityData to 7 Oct 2026

Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions

arXiv 2105.04471 ↗First posted 10 May 2021cs.LG stat.MLcited by 13

Not on the map. Why?

Abstract (arXiv)

Uncertainty awareness is crucial to develop reliable machine learning models. In this work, we propose the Natural Posterior Network (NatPN) for fast and high-quality uncertainty estimation for any task where the target distribution belongs to the exponential family. Thus, NatPN finds application for both classification and general regression settings. Unlike many previous approaches, NatPN does not require out-of-distribution (OOD) data at training time. Instead, it leverages Normalizing Flows to fit a single density on a learned low-dimensional and task-dependent latent space. For any input sample, NatPN uses the predicted likelihood to perform a Bayesian update over the target distribution. Theoretically, NatPN assigns high uncertainty far away from training data. Empirically, our extensive experiments on calibration and OOD detection show that NatPN delivers highly competitive performance for classification, regression and count prediction tasks.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 57 of its 95 references that are in the map. First posted on arXiv 10 May 2021.

Builds on

57 of 95 references.

References · 57

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  8. 2015Variational Inference with Normalizing Flowscited by 2,161
  9. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  10. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  11. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  12. 2017Masked Autoregressive Flow for Density Estimationcited by 680
38 more
  1. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  2. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  3. 2015Weight Uncertainty in Neural Networkscited by 554
  4. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  5. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
  6. 2018Deep Learning for Classical Japanese Literaturecited by 415
  7. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  8. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  9. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  10. 2019Deep Evidential Regressioncited by 354
  11. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  12. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  13. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  14. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  15. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 253
  16. 2017Advances in Variational Inferencecited by 236
  17. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 220
  18. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  19. 2013A General Framework for Updating Belief Distributionscited by 172
  20. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  21. 2020Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Countscited by 163
  22. 2019Ensemble Distribution Distillationcited by 158
  23. 2019Practical Deep Learning with Bayesian Principlescited by 158
  24. 2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 149
  25. 2020Confidence-Aware Learning for Deep Neural Networkscited by 113
  26. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  27. 2020Uncertainty Aware Semi-Supervised Learning on Graph Datacited by 95
  28. 2019Distribution Calibration for Regressioncited by 88
  29. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  30. 2018Lightweight Probabilistic Deep Networkscited by 77
  31. 2019Sampling-free Epistemic Uncertainty Estimation Using Approximated Variance Propagationcited by 74
  32. 2020Density of States Estimation for Out-of-Distribution Detectioncited by 74
  33. 2019Towards neural networks that provably know when they don't knowcited by 69
  34. 2020Intra Order-preserving Functions for Calibration of Multi-Class Neural Networkscited by 65
  35. 2019A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Taskscited by 62
  36. 2020Individual Calibration with Randomized Forecastingcited by 50
  37. 2020Uncertainty-Aware Deep Classifiers using Generative Modelscited by 41
  38. 2020Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examplescited by 40

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 6; 2025: 4.

13 papers.

Sort:

By area:

  1. 2024Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Modelscited by 22
  2. 2024Uncertainty Estimation by Density Aware Evidential Deep Learningcited by 22
  3. 2024Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage?cited by 17
  4. 2024Revisiting Essential and Nonessential Settings of Evidential Deep Learningcited by 15
  5. 2023Uncertainty Estimation for Molecules: Desiderata and Methodscited by 11
  6. 2025Position: Epistemic uncertainty estimation methods are fundamentally incompletecited by 8
  7. 2023Topology-Matching Normalizing Flows for Out-of-Distribution Detection in Robot Learningcited by 7
  8. 2024A Unified Approach Towards Active Learning and Out-of-Distribution Detectioncited by 7
  9. 2025Prior2Former -- Evidential Modeling of Mask Transformers for Assumption-Free Open-World Panoptic Segmentationcited by 7
  10. 2025Joint Out-of-Distribution Filtering and Data Discovery Active Learningcited by 6
  11. 2024Discriminant Distance-Aware Representation on Deterministic Uncertainty Quantification Methodscited by 2
  12. 2025PRESCRIBE: Predicting Single-Cell Responses with Bayesian Estimationcited by 2
1 more
  1. 2023Efficient Epistemic Uncertainty Estimation in Regression Ensemble Models Using Pairwise-Distance Estimatorscited by 1