# Bayesian Hypernetworks

arXiv: https://arxiv.org/abs/1710.04759 · HTML: https://mlsanity.com/paper/1710.04759/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2017.

## Built on by (most cited first, 30 of 66)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Permutation Equivariant Neural Functionals](https://mlsanity.com/paper/2302.14040/) (2023)
- [Massive Editing for Large Language Models via Meta Learning](https://mlsanity.com/paper/2311.04661/) (2023)
- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [Estimating Epistemic and Aleatoric Uncertainty with a Single Model](https://mlsanity.com/paper/2402.03478/) (2024)
- [Neural Functional Transformers](https://mlsanity.com/paper/2305.13546/) (2023)
- [Posterior Meta-Replay for Continual Learning](https://mlsanity.com/paper/2103.01133/) (2021)
- [Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations](https://mlsanity.com/paper/2102.06559/) (2021)
- [Learning the Effect of Registration Hyperparameters with HyperMorph](https://mlsanity.com/paper/2203.16680/) (2022)
- [Universal Neural Functionals](https://mlsanity.com/paper/2402.05232/) (2024)
- [FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation](https://mlsanity.com/paper/2310.00339/) (2023)
- [HyperFast: Instant Classification for Tabular Data](https://mlsanity.com/paper/2402.14335/) (2024)
- [Hypernetworks for Zero-shot Transfer in Reinforcement Learning](https://mlsanity.com/paper/2211.15457/) (2022)
- [Principled Weight Initialization for Hypernetworks](https://mlsanity.com/paper/2312.08399/) (2023)
- [Example-based Hypernetworks for Out-of-Distribution Generalization](https://mlsanity.com/paper/2203.14276/) (2022)
- [Boosting Natural Language Generation from Instructions with Meta-Learning](https://mlsanity.com/paper/2210.11617/) (2022)
- [Amortized In-Context Bayesian Posterior Estimation](https://mlsanity.com/paper/2502.06601/) (2025)
- [Magnitude Invariant Parametrizations Improve Hypernetwork Learning](https://mlsanity.com/paper/2304.07645/) (2023)
- [Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification](https://mlsanity.com/paper/2405.15047/) (2024)
- [When Meta-Learning Meets Online and Continual Learning: A Survey](https://mlsanity.com/paper/2311.05241/) (2023)
- [Are Bayesian neural networks intrinsically good at out-of-distribution detection?](https://mlsanity.com/paper/2107.12248/) (2021)
- [Regularization-Agnostic Compressed Sensing MRI Reconstruction with Hypernetworks](https://mlsanity.com/paper/2101.02194/) (2021)
- [Meta-Learning via Classifier(-free) Diffusion Guidance](https://mlsanity.com/paper/2210.08942/) (2022)
- [Computing Multiple Image Reconstructions with a Single Hypernetwork](https://mlsanity.com/paper/2202.11009/) (2022)
- [Multi-Task Neural Processes](https://mlsanity.com/paper/2111.05820/) (2021)
- [Generative Hyperelasticity with Physics-Informed Probabilistic Diffusion Fields](https://mlsanity.com/paper/2310.03745/) (2023)
- [Learning Credal Ensembles via Distributionally Robust Optimization](https://mlsanity.com/paper/2602.08470/) (2026)

## Abstract (arXiv)

We study Bayesian hypernetworks: a framework for approximate Bayesian inference in neural networks. A Bayesian hypernetwork h is a neural network which learns to transform a simple noise distribution, p(vecε) = N(vec 0,mat I), to a distribution q(pp) := q(h(vecε)) over the parameters pp of another neural network (the "primary network")@. We train q with variational inference, using an invertible h to enable efficient estimation of the variational lower bound on the posterior p(pp | D) via sampling. In contrast to most methods for Bayesian deep learning, Bayesian hypernets can represent a complex multimodal approximate posterior with correlations between parameters, while enabling cheap iid sampling of~q(pp). In practice, Bayesian hypernets can provide a better defense against adversarial examples than dropout, and also exhibit competitive performance on a suite of tasks which evaluate model uncertainty, including regularization, active learning, and anomaly detection.