# Implicit Weight Uncertainty in Neural Networks

arXiv: https://arxiv.org/abs/1711.01297 · HTML: https://mlsanity.com/paper/1711.01297/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2017.

## Built on by (most cited first, 30 of 30)

- [Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matters](https://mlsanity.com/paper/2205.13703/) (2022)
- [Estimating Epistemic and Aleatoric Uncertainty with a Single Model](https://mlsanity.com/paper/2402.03478/) (2024)
- [Posterior Meta-Replay for Continual Learning](https://mlsanity.com/paper/2103.01133/) (2021)
- [LiBRe: A Practical Bayesian Approach to Adversarial Detection](https://mlsanity.com/paper/2103.14835/) (2021)
- [HyperDeepONet: learning operator with complex target function space using the limited resources via hypernetwork](https://mlsanity.com/paper/2312.15949/) (2023)
- [Principled Weight Initialization for Hypernetworks](https://mlsanity.com/paper/2312.08399/) (2023)
- [Bayesian OOD detection with aleatoric uncertainty and outlier exposure](https://mlsanity.com/paper/2102.12959/) (2021)
- [Example-based Hypernetworks for Out-of-Distribution Generalization](https://mlsanity.com/paper/2203.14276/) (2022)
- [A Bayesian neural network predicts the dissolution of compact planetary systems](https://mlsanity.com/paper/2101.04117/) (2021)
- [Magnitude Invariant Parametrizations Improve Hypernetwork Learning](https://mlsanity.com/paper/2304.07645/) (2023)
- [In Defense of the Learning Without Forgetting for Task Incremental Learning](https://mlsanity.com/paper/2107.12304/) (2021)
- [Are Bayesian neural networks intrinsically good at out-of-distribution detection?](https://mlsanity.com/paper/2107.12248/) (2021)
- [Safety Enhancement for Deep Reinforcement Learning in Autonomous Separation Assurance](https://mlsanity.com/paper/2105.02331/) (2021)
- [Learning Unbiased Transferability for Domain Adaptation by Uncertainty Modeling](https://mlsanity.com/paper/2206.01319/) (2022)
- [HyperSINDy: Deep Generative Modeling of Nonlinear Stochastic Governing Equations](https://mlsanity.com/paper/2310.04832/) (2023)
- [On out-of-distribution detection with Bayesian neural networks](https://mlsanity.com/paper/2110.06020/) (2021)
- [Block-local learning with probabilistic latent representations](https://mlsanity.com/paper/2305.14974/) (2023)
- [Implicit Variational Inference for High-Dimensional Posteriors](https://mlsanity.com/paper/2310.06643/) (2023)
- [Adversarial Attack for Uncertainty Estimation: Identifying Critical Regions in Neural Networks](https://mlsanity.com/paper/2107.07618/) (2021)
- [Time Series Forecasting with Hypernetworks Generating Parameters in Advance](https://mlsanity.com/paper/2211.12034/) (2022)
- [Uncertainty-Aware Rank-One MIMO Q Network Framework for Accelerated Offline Reinforcement Learning](https://mlsanity.com/paper/2602.19917/) (2026)
- [Distributional Gaussian Processes Layers for Out-of-Distribution Detection](https://mlsanity.com/paper/2206.13346/) (2022)
- [Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Temporal Grounding](https://mlsanity.com/paper/2508.06317/) (2025)
- [Generative Particle Variational Inference via Estimation of Functional Gradients](https://mlsanity.com/paper/2103.01291/) (2021)
- [Graceful Degradation and Related Fields](https://mlsanity.com/paper/2106.11119/) (2021)
- [Bayesian Neural Network Inference via Implicit Models and the Posterior Predictive Distribution](https://mlsanity.com/paper/2209.02188/) (2022)
- [Scale-Space Hypernetworks for Efficient Biomedical Imaging](https://mlsanity.com/paper/2304.05448/) (2023)
- [Digital Twin Framework for Optimal and Autonomous Decision-Making in Cyber-Physical Systems: Enhancing Reliability and Adaptability in the Oil and Gas Industry](https://mlsanity.com/paper/2311.12755/) (2023)
- [LatentTrack: Sequential Weight Generation via Latent Filtering](https://mlsanity.com/paper/2602.00458/) (2026)
- [Learning to Predict Distributions over Weight Updates for Test-Time Adaptation](https://mlsanity.com/paper/2610.01934/) (2026)

## Abstract (arXiv)

Modern neural networks tend to be overconfident on unseen, noisy or incorrectly labelled data and do not produce meaningful uncertainty measures. Bayesian deep learning aims to address this shortcoming with variational approximations (such as Bayes by Backprop or Multiplicative Normalising Flows). However, current approaches have limitations regarding flexibility and scalability. We introduce Bayes by Hypernet (BbH), a new method of variational approximation that interprets hypernetworks as implicit distributions. It naturally uses neural networks to model arbitrarily complex distributions and scales to modern deep learning architectures. In our experiments, we demonstrate that our method achieves competitive accuracies and predictive uncertainties on MNIST and a CIFAR5 task, while being the most robust against adversarial attacks.