# HyperGAN: A Generative Model for Diverse, Performant Neural Networks

arXiv: https://arxiv.org/abs/1901.11058 · HTML: https://mlsanity.com/paper/1901.11058/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2019.

## Built on by (most cited first, 30 of 51)

- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [Parameter Prediction for Unseen Deep Architectures](https://mlsanity.com/paper/2110.13100/) (2021)
- [Learning to Learn with Generative Models of Neural Network Checkpoints](https://mlsanity.com/paper/2209.12892/) (2022)
- [Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization](https://mlsanity.com/paper/2105.05612/) (2021)
- [Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weights](https://mlsanity.com/paper/2209.14733/) (2022)
- [HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning](https://mlsanity.com/paper/2201.04182/) (2022)
- [Towards Scalable and Versatile Weight Space Learning](https://mlsanity.com/paper/2406.09997/) (2024)
- [Model Zoos: A Dataset of Diverse Populations of Neural Network Models](https://mlsanity.com/paper/2209.14764/) (2022)
- [Diffusion-Based Neural Network Weights Generation](https://mlsanity.com/paper/2402.18153/) (2024)
- [Recomposing the Reinforcement Learning Building Blocks with Hypernetworks](https://mlsanity.com/paper/2106.06842/) (2021)
- [Learning the Effect of Registration Hyperparameters with HyperMorph](https://mlsanity.com/paper/2203.16680/) (2022)
- [Conditional Denoising Diffusion for Sequential Recommendation](https://mlsanity.com/paper/2304.11433/) (2023)
- [HyperFast: Instant Classification for Tabular Data](https://mlsanity.com/paper/2402.14335/) (2024)
- [HyperINR: A Fast and Predictive Hypernetwork for Implicit Neural Representations via Knowledge Distillation](https://mlsanity.com/paper/2304.04188/) (2023)
- [Principled Weight Initialization for Hypernetworks](https://mlsanity.com/paper/2312.08399/) (2023)
- [HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworks](https://mlsanity.com/paper/2103.09439/) (2021)
- [DynaGAN: Dynamic Few-shot Adaptation of GANs to Multiple Domains](https://mlsanity.com/paper/2211.14554/) (2022)
- [Magnitude Invariant Parametrizations Improve Hypernetwork Learning](https://mlsanity.com/paper/2304.07645/) (2023)
- [Federated Learning with Uncertainty via Distilled Predictive Distributions](https://mlsanity.com/paper/2206.07562/) (2022)
- [Hyper-Representations for Pre-Training and Transfer Learning](https://mlsanity.com/paper/2207.10951/) (2022)
- [ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generation](https://mlsanity.com/paper/2402.12408/) (2024)
- [CLIPtone: Unsupervised Learning for Text-based Image Tone Adjustment](https://mlsanity.com/paper/2404.01123/) (2024)
- [Dense Uncertainty Estimation](https://mlsanity.com/paper/2110.06427/) (2021)
- [Meta-Learning via Classifier(-free) Diffusion Guidance](https://mlsanity.com/paper/2210.08942/) (2022)
- [XReal: Realistic Anatomy and Pathology-Aware X-ray Generation via Controllable Diffusion Model](https://mlsanity.com/paper/2403.09240/) (2024)
- [Meta Internal Learning](https://mlsanity.com/paper/2110.02900/) (2021)
- [HyperNeRFGAN: Hypernetwork approach to 3D NeRF GAN](https://mlsanity.com/paper/2301.11631/) (2023)
- [Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Survey](https://mlsanity.com/paper/2404.05219/) (2024)
- [Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstruction](https://mlsanity.com/paper/2503.17138/) (2025)
- [DeepWeightFlow: Re-Basined Flow Matching for Generating Neural Network Weights](https://mlsanity.com/paper/2601.05052/) (2026)

## Abstract (arXiv)

Standard neural networks are often overconfident when presented with data outside the training distribution. We introduce HyperGAN, a new generative model for learning a distribution of neural network parameters. HyperGAN does not require restrictive assumptions on priors, and networks sampled from it can be used to quickly create very large and diverse ensembles. HyperGAN employs a novel mixer to project prior samples to a latent space with correlated dimensions, and samples from the latent space are then used to generate weights for each layer of a deep neural network. We show that HyperGAN can learn to generate parameters which label the MNIST and CIFAR-10 datasets with competitive performance to fully supervised learning, while learning a rich distribution of effective parameters. We also show that HyperGAN can also provide better uncertainty estimates than standard ensembles by evaluating on out of distribution data as well as adversarial examples.