# Variational Gaussian Dropout is not Bayesian

arXiv: https://arxiv.org/abs/1711.02989 · HTML: https://mlsanity.com/paper/1711.02989/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2017.

## Built on by (most cited first, 13 of 13)

- [Epistemic Neural Networks](https://mlsanity.com/paper/2107.08924/) (2021)
- [The Neural Testbed: Evaluating Joint Predictions](https://mlsanity.com/paper/2110.04629/) (2021)
- [Fine-Tuning Language Models via Epistemic Neural Networks](https://mlsanity.com/paper/2211.01568/) (2022)
- [Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspective](https://mlsanity.com/paper/2401.08667/) (2024)
- [On the Effects of Quantisation on Model Uncertainty in Bayesian Neural Networks](https://mlsanity.com/paper/2102.11062/) (2021)
- [Conformalized Unconditional Quantile Regression](https://mlsanity.com/paper/2304.01426/) (2023)
- [Towards Reliable Uncertainty Quantification via Deep Ensembles in Multi-output Regression Task](https://mlsanity.com/paper/2303.16210/) (2023)
- [Sparsifying Bayesian neural networks with latent binary variables and normalizing flows](https://mlsanity.com/paper/2305.03395/) (2023)
- [Neural Variational Dropout Processes](https://mlsanity.com/paper/2510.19425/) (2025)
- [Estimating the Uncertainty of Neural Network Forecasts for Influenza Prevalence Using Web Search Activity](https://mlsanity.com/paper/2105.12433/) (2021)
- [Epinet for Content Cold Start](https://mlsanity.com/paper/2412.04484/) (2024)
- [Centroid Approximation for Bootstrap: Improving Particle Quality at Inference](https://mlsanity.com/paper/2110.08720/) (2021)
- [Forecasting infectious disease prevalence with associated uncertainty using neural networks](https://mlsanity.com/paper/2409.01154/) (2024)

## Abstract (arXiv)

Gaussian multiplicative noise is commonly used as a stochastic regularisation technique in training of deterministic neural networks. A recent paper reinterpreted the technique as a specific algorithm for approximate inference in Bayesian neural networks; several extensions ensued. We show that the log-uniform prior used in all the above publications does not generally induce a proper posterior, and thus Bayesian inference in such models is ill-posed. Independent of the log-uniform prior, the correlated weight noise approximation has further issues leading to either infinite objective or high risk of overfitting. The above implies that the reported sparsity of obtained solutions cannot be explained by Bayesian or the related minimum description length arguments. We thus study the objective from a non-Bayesian perspective, provide its previously unknown analytical form which allows exact gradient evaluation, and show that the later proposed additive reparametrisation introduces minima not present in the original multiplicative parametrisation. Implications and future research directions are discussed.