# Dropout Inference in Bayesian Neural Networks with Alpha-divergences

arXiv: https://arxiv.org/abs/1703.02914 · HTML: https://mlsanity.com/paper/1703.02914/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Mar 2017.

## Built on by (most cited first, 28 of 28)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [LiBRe: A Practical Bayesian Approach to Adversarial Detection](https://mlsanity.com/paper/2103.14835/) (2021)
- [Predictive Inference with Feature Conformal Prediction](https://mlsanity.com/paper/2210.00173/) (2022)
- [A Comprehensive Survey on Evidential Deep Learning and Its Applications](https://mlsanity.com/paper/2409.04720/) (2024)
- [Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learning](https://mlsanity.com/paper/2205.06342/) (2022)
- [Variational Linearized Laplace Approximation for Bayesian Deep Learning](https://mlsanity.com/paper/2302.12565/) (2023)
- [Out-of-Distribution Detection for LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2209.14435/) (2022)
- [Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2205.07708/) (2022)
- [Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptotics](https://mlsanity.com/paper/2210.06226/) (2022)
- [On the Robustness of Bayesian Neural Networks to Adversarial Attacks](https://mlsanity.com/paper/2207.06154/) (2022)
- [Particle-based Online Bayesian Sampling](https://mlsanity.com/paper/2302.14796/) (2023)
- [Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs](https://mlsanity.com/paper/2105.06512/) (2021)
- [Bayesian posterior approximation with stochastic ensembles](https://mlsanity.com/paper/2212.08123/) (2022)
- [On the use of uncertainty in classifying Aedes Albopictus mosquitoes](https://mlsanity.com/paper/2110.15912/) (2021)
- [On permutation symmetries in Bayesian neural network posteriors: a variational perspective](https://mlsanity.com/paper/2310.10171/) (2023)
- [E-PINNs: Epistemic Physics-Informed Neural Networks](https://mlsanity.com/paper/2503.19333/) (2025)
- [Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversity](https://mlsanity.com/paper/2512.05962/) (2025)
- [Variational Routing: A Scalable Bayesian Framework for Calibrated Mixture-of-Experts Transformers](https://mlsanity.com/paper/2603.09453/) (2026)
- [Fixed-Mean Gaussian Processes for Post-hoc Bayesian Deep Learning](https://mlsanity.com/paper/2412.04177/) (2024)
- [Estimating the Uncertainty of Neural Network Forecasts for Influenza Prevalence Using Web Search Activity](https://mlsanity.com/paper/2105.12433/) (2021)
- [BADDr: Bayes-Adaptive Deep Dropout RL for POMDPs](https://mlsanity.com/paper/2202.08884/) (2022)
- [Alpha Entropy Search for New Information-based Bayesian Optimization](https://mlsanity.com/paper/2411.16586/) (2024)
- [OFAL: An Oracle-Free Active Learning Framework](https://mlsanity.com/paper/2508.08126/) (2025)
- [Laplace Approximation for Bayesian Tensor Network Kernel Machines](https://mlsanity.com/paper/2604.26673/) (2026)
- [Excess risk analysis for epistemic uncertainty with application to variational inference](https://mlsanity.com/paper/2206.01606/) (2022)
- [Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding](https://mlsanity.com/paper/2605.08446/) (2026)
- [Soft Specialists: α-R'enyi Ensembles for Uncertainty-Aware LLM Post-Training](https://mlsanity.com/paper/2605.27747/) (2026)
- [Uncertainty quantification for trustworthy deep learning: Methods and measures](https://mlsanity.com/paper/2607.28248/) (2026)

## Abstract (arXiv)

To obtain uncertainty estimates with real-world Bayesian deep learning models, practical inference approximations are needed. Dropout variational inference (VI) for example has been used for machine vision and medical applications, but VI can severely underestimates model uncertainty. Alpha-divergences are alternative divergences to VI's KL objective, which are able to avoid VI's uncertainty underestimation. But these are hard to use in practice: existing techniques can only use Gaussian approximating distributions, and require existing models to be changed radically, thus are of limited use for practitioners. We propose a re-parametrisation of the alpha-divergence objectives, deriving a simple inference technique which, together with dropout, can be easily implemented with existing models by simply changing the loss of the model. We demonstrate improved uncertainty estimates and accuracy compared to VI in dropout networks. We study our model's epistemic uncertainty far away from the data using adversarial images, showing that these can be distinguished from non-adversarial images by examining our model's uncertainty.