# Overpruning in Variational Bayesian Neural Networks

arXiv: https://arxiv.org/abs/1801.06230 · HTML: https://mlsanity.com/paper/1801.06230/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2018.

## Built on by (most cited first, 22 of 22)

- [Bayesian Neural Network Priors Revisited](https://mlsanity.com/paper/2102.06571/) (2021)
- [Variational Learning is Effective for Large Deep Networks](https://mlsanity.com/paper/2402.17641/) (2024)
- [Do Bayesian Neural Networks Need To Be Fully Stochastic?](https://mlsanity.com/paper/2211.06291/) (2022)
- [BayesDAG: Gradient-Based Posterior Inference for Causal Discovery](https://mlsanity.com/paper/2307.13917/) (2023)
- [A survey on Variational Autoencoders from a GreenAI perspective](https://mlsanity.com/paper/2103.01071/) (2021)
- [Wide Mean-Field Bayesian Neural Networks Ignore the Data](https://mlsanity.com/paper/2202.11670/) (2022)
- [Partitioned Variational Inference: A Framework for Probabilistic Federated Learning](https://mlsanity.com/paper/2202.12275/) (2022)
- [RECOMBINER: Robust and Enhanced Compression with Bayesian Implicit Neural Representations](https://mlsanity.com/paper/2309.17182/) (2023)
- [On the detrimental effect of invariances in the likelihood for variational inference](https://mlsanity.com/paper/2209.07157/) (2022)
- [Wide Mean-Field Variational Bayesian Neural Networks Ignore the Data](https://mlsanity.com/paper/2106.07052/) (2021)
- [TyXe: Pyro-based Bayesian neural nets for Pytorch](https://mlsanity.com/paper/2110.00276/) (2021)
- [Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradients](https://mlsanity.com/paper/2206.06295/) (2022)
- [Variational Visual Question Answering for Uncertainty-Aware Selective Prediction](https://mlsanity.com/paper/2505.09591/) (2025)
- [Density Uncertainty Layers for Reliable Uncertainty Estimation](https://mlsanity.com/paper/2306.12497/) (2023)
- [On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors](https://mlsanity.com/paper/2603.22030/) (2026)
- [Enhancing variational generation through self-decomposition](https://mlsanity.com/paper/2202.02738/) (2022)
- [Leveraging Task Structures for Improved Identifiability in Neural Network Representations](https://mlsanity.com/paper/2306.14861/) (2023)
- [Temporal-Difference Variational Continual Learning](https://mlsanity.com/paper/2410.07812/) (2024)
- [BALI: Learning Neural Networks via Bayesian Layerwise Inference](https://mlsanity.com/paper/2411.12102/) (2024)
- [Understanding the Trade-offs in Accuracy and Uncertainty Quantification: Architecture and Inference Choices in Bayesian Neural Networks](https://mlsanity.com/paper/2503.11808/) (2025)
- [Calibrated Sampling-Free Uncertainty Estimation in Bayesian Deep Learning](https://mlsanity.com/paper/2606.16214/) (2026)
- [The Impact of Likelihood Tempering on the Limiting Predictive Moments of Variational Bayesian Linear Neural Networks](https://mlsanity.com/paper/2610.09132/) (2026)

## Abstract (arXiv)

The motivations for using variational inference (VI) in neural networks differ significantly from those in latent variable models. This has a counter-intuitive consequence; more expressive variational approximations can provide significantly worse predictions as compared to those with less expressive families. In this work we make two contributions. First, we identify a cause of this performance gap, variational over-pruning. Second, we introduce a theoretically grounded explanation for this phenomenon. Our perspective sheds light on several related published results and provides intuition into the design of effective variational approximations of neural networks.