# BNNpriors: A library for Bayesian neural network inference with different prior distributions

arXiv: https://arxiv.org/abs/2105.06964 · HTML: https://mlsanity.com/paper/2105.06964/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 14 May 2021.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors](https://mlsanity.com/paper/2005.07186/) (2020)
- [Bayesian Neural Network Priors Revisited](https://mlsanity.com/paper/2102.06571/) (2021)
- [Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning](https://mlsanity.com/paper/2104.04975/) (2021)
- [PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees](https://mlsanity.com/paper/2002.05551/) (2020)
- [Marginal likelihood computation for model selection and hypothesis testing: an extensive review](https://mlsanity.com/paper/2005.08334/) (2020)
- [Exact Langevin Dynamics with Stochastic Gradients](https://mlsanity.com/paper/2102.01691/) (2021)

## Built on by (most cited first, 20 of 20)

- [Repulsive Deep Ensembles are Bayesian](https://mlsanity.com/paper/2106.11642/) (2021)
- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [Credal Bayesian Deep Learning](https://mlsanity.com/paper/2302.09656/) (2023)
- [Data augmentation in Bayesian neural networks and the cold posterior effect](https://mlsanity.com/paper/2106.05586/) (2021)
- [On Stein Variational Neural Network Ensembles](https://mlsanity.com/paper/2106.10760/) (2021)
- [Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2405.03425/) (2024)
- [Neural Variational Gradient Descent](https://mlsanity.com/paper/2107.10731/) (2021)
- [Improving Neural Additive Models with Bayesian Principles](https://mlsanity.com/paper/2305.16905/) (2023)
- [Pathologies in priors and inference for Bayesian transformers](https://mlsanity.com/paper/2110.04020/) (2021)
- [Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization](https://mlsanity.com/paper/2304.08309/) (2023)
- [Incorporating Unlabelled Data into Bayesian Neural Networks](https://mlsanity.com/paper/2304.01762/) (2023)
- [Deep Classifiers with Label Noise Modeling and Distance Awareness](https://mlsanity.com/paper/2110.02609/) (2021)
- [Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metrics](https://mlsanity.com/paper/2303.05101/) (2023)
- [TyXe: Pyro-based Bayesian neural nets for Pytorch](https://mlsanity.com/paper/2110.00276/) (2021)
- [Graph Evidential Learning for Anomaly Detection](https://mlsanity.com/paper/2506.00594/) (2025)
- [Bayesian Physics Informed Neural Networks for Reliable Transformer Prognostics](https://mlsanity.com/paper/2509.15933/) (2025)
- [A Bayesian Approach to Invariant Deep Neural Networks](https://mlsanity.com/paper/2107.09301/) (2021)
- [Uncertainty in Graph Contrastive Learning with Bayesian Neural Networks](https://mlsanity.com/paper/2312.00232/) (2023)
- [Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics](https://mlsanity.com/paper/2601.03673/) (2026)
- [Function-Space Empirical Bayes Regularisation with Student's t Priors](https://mlsanity.com/paper/2602.22015/) (2026)

## Abstract (arXiv)

Bayesian neural networks have shown great promise in many applications where calibrated uncertainty estimates are crucial and can often also lead to a higher predictive performance. However, it remains challenging to choose a good prior distribution over their weights. While isotropic Gaussian priors are often chosen in practice due to their simplicity, they do not reflect our true prior beliefs well and can lead to suboptimal performance. Our new library, BNNpriors, enables state-of-the-art Markov Chain Monte Carlo inference on Bayesian neural networks with a wide range of predefined priors, including heavy-tailed ones, hierarchical ones, and mixture priors. Moreover, it follows a modular approach that eases the design and implementation of new custom priors. It has facilitated foundational discoveries on the nature of the cold posterior effect in Bayesian neural networks and will hopefully catalyze future research as well as practical applications in this area.