# Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling

arXiv: https://arxiv.org/abs/2210.00993 · HTML: https://mlsanity.com/paper/2210.00993/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 46 references that are in the map.
- First posted on arXiv 3 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [MNIST-C: A Robustness Benchmark for Computer Vision](https://mlsanity.com/paper/1906.02337/) (2019)
- [Functional Variational Bayesian Neural Networks](https://mlsanity.com/paper/1903.05779/) (2019)
- [Efficient Bayesian computation by proximal Markov chain Monte Carlo: when Langevin meets Moreau](https://mlsanity.com/paper/1612.07471/) (2016)
- [Pareto Smoothed Importance Sampling](https://mlsanity.com/paper/1507.02646/) (2015)
- [Adversarial Distillation of Bayesian Neural Network Posteriors](https://mlsanity.com/paper/1806.10317/) (2018)
- [Layered Adaptive Importance Sampling](https://mlsanity.com/paper/1505.04732/) (2015)
- [Convergence rates for optimised adaptive importance samplers](https://mlsanity.com/paper/1903.12044/) (2019)
- [Gradient Importance Sampling](https://mlsanity.com/paper/1507.05781/) (2015)
- [Langevin Incremental Mixture Importance Sampling](https://mlsanity.com/paper/1611.06874/) (2016)

## Built on by (most cited first, 2 of 2)

- [A Primer on Bayesian Neural Networks: Review and Debates](https://mlsanity.com/paper/2309.16314/) (2023)
- [Stein Variational Newton Neural Network Ensembles](https://mlsanity.com/paper/2411.01887/) (2024)

## Abstract (arXiv)

Bayesian neural networks (BNNs) have received an increased interest in the last years. In BNNs, a complete posterior distribution of the unknown weight and bias parameters of the network is produced during the training stage. This probabilistic estimation offers several advantages with respect to point-wise estimates, in particular, the ability to provide uncertainty quantification when predicting new data. This feature inherent to the Bayesian paradigm, is useful in countless machine learning applications. It is particularly appealing in areas where decision-making has a crucial impact, such as medical healthcare or autonomous driving. The main challenge of BNNs is the computational cost of the training procedure since Bayesian techniques often face a severe curse of dimensionality. Adaptive importance sampling (AIS) is one of the most prominent Monte Carlo methodologies benefiting from sounded convergence guarantees and ease for adaptation. This work aims to show that AIS constitutes a successful approach for designing BNNs. More precisely, we propose a novel algorithm PMCnet that includes an efficient adaptation mechanism, exploiting geometric information on the complex (often multimodal) posterior distribution. Numerical results illustrate the excellent performance and the improved exploration capabilities of the proposed method for both shallow and deep neural networks.