mlsanityData to 7 Oct 2026

HyperNetworks

arXiv 1609.09106 ↗First posted 27 Sep 2016cs.LGcited by 1,003

Not on the map. Why?

Cited by 52 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This work explores hypernetworks: an approach of using a one network, also known as a hypernetwork, to generate the weights for another network. Hypernetworks provide an abstraction that is similar to what is found in nature: the relationship between a genotype - the hypernetwork - and a phenotype - the main network. Though they are also reminiscent of HyperNEAT in evolution, our hypernetworks are trained end-to-end with backpropagation and thus are usually faster. The focus of this work is to make hypernetworks useful for deep convolutional networks and long recurrent networks, where hypernetworks can be viewed as relaxed form of weight-sharing across layers. Our main result is that hypernetworks can generate non-shared weights for LSTM and achieve near state-of-the-art results on a variety of sequence modelling tasks including character-level language modelling, handwriting generation and neural machine translation, challenging the weight-sharing paradigm for recurrent networks. Our results also show that hypernetworks applied to convolutional networks still achieve respectable results for image recognition tasks compared to state-of-the-art baseline models while requiring fewer learnable parameters.

Key facts. Cited by 1,003 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 137; 2022: 160; 2023: 207; 2024: 172; 2025: 167; 2026: 159.

1,003 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  3. 2021Editing Factual Knowledge in Language Modelscited by 548
  4. 2021Neural Fields in Visual Computing and Beyondcited by 539
  5. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  6. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  7. 2021Fast Model Editing at Scalecited by 491
  8. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  9. 2021Dynamic Neural Networks: A Surveycited by 405
  10. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  11. 2021GAN Inversion: A Surveycited by 366
  12. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
38 more
  1. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  2. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  3. 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 278
  4. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
  5. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  6. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  7. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  8. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wildcited by 197
  9. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  10. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  11. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  12. 2021Modulated Periodic Activations for Generalizable Local Functional Representationscited by 140
  13. 2021Local Texture Estimator for Implicit Representation Functioncited by 126
  14. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  15. 2021LambdaNetworks: Modeling Long-Range Interactions Without Attentioncited by 107
  16. 2021Stylizing 3D Scene via Implicit Representation and HyperNetworkcited by 103
  17. 2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 102
  18. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  19. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  20. 2021Involution: Inverting the Inherence of Convolution for Visual Recognitioncited by 99
  21. 2024Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observationcited by 97
  22. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  23. 2022Spatio-Temporal Meta-Graph Learning for Traffic Forecastingcited by 88
  24. 2021HyperInverter: Improving StyleGAN Inversion via Hypernetworkcited by 85
  25. 2021HyperMorph: Amortized Hyperparameter Learning for Image Registrationcited by 84
  26. 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
  27. 2022NOMAD: Nonlinear Manifold Decoders for Operator Learningcited by 81
  28. 2023Learning to Search Feasible and Infeasible Regions of Routing Problems with Flexible Neural k-Optcited by 81
  29. 2021Top-KAST: Top-K Always Sparse Trainingcited by 79
  30. 2022Transformers as Meta-Learners for Implicit Neural Representationscited by 79
  31. 2023Effective Real Image Editing with Accelerated Iterative Diffusion Inversioncited by 79
  32. 2022Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracingcited by 78
  33. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  34. 2022Continuous PDE Dynamics Forecasting with Implicit Neural Representationscited by 71
  35. 2021Aligning Latent and Image Spaces to Connect the Unconnectablecited by 70
  36. 2023Permutation Equivariant Neural Functionalscited by 69
  37. 2022HyperPrompt: Prompt-based Task-Conditioning of Transformerscited by 67
  38. 2022Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weightscited by 63

50 most cited of 1,003 All 1,003 (JSON)