mlsanityData to 14 Oct 2026

Graph HyperNetworks for Neural Architecture Search

arXiv 1810.05749 ↗First posted 12 Oct 2018cs.LG cs.CV stat.MLcited by 132

Not on the map. Why?

Abstract (arXiv)

Neural architecture search (NAS) automatically finds the best task-specific neural network topology, outperforming many manual architecture designs. However, it can be prohibitively expensive as the search requires training thousands of different networks, while each can last for hours. In this work, we propose the Graph HyperNetwork (GHN) to amortize the search cost: given an architecture, it directly generates the weights by running inference on a graph neural network. GHNs model the topology of an architecture and therefore can predict network performance more accurately than regular hypernetworks and premature early stopping. To perform NAS, we randomly sample architectures and use the validation accuracy of networks with GHN generated weights as the surrogate search signal. GHNs are fast -- they can search nearly 10 times faster than other random search methods on CIFAR-10 and ImageNet. GHNs can be further extended to the anytime prediction setting, where they have found networks with better speed-accuracy tradeoff than the state-of-the-art manual designs.

Key facts. Cited by 132 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 22; 2022: 29; 2023: 27; 2024: 20; 2025: 23; 2026: 11.

132 papers.

Sort:

By area:

  1. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  2. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  3. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  4. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  5. 2023Deep Model Fusion: A Surveycited by 91
  6. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  7. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  8. 2021HyperInverter: Improving StyleGAN Inversion via Hypernetworkcited by 85
  9. 2022Tutorial on amortized optimizationcited by 83
  10. 2023Permutation Equivariant Neural Functionalscited by 70
  11. 2022Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weightscited by 64
  12. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 58
38 more
  1. 2023Neural Functional Transformerscited by 52
  2. 2024Towards Scalable and Versatile Weight Space Learningcited by 41
  3. 2021Automated Machine Learning on Graphs: A Surveycited by 39
  4. 2022Model Zoos: A Dataset of Diverse Populations of Neural Network Modelscited by 39
  5. 2024Diffusion-Based Neural Network Weights Generationcited by 39
  6. 2022DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalizationcited by 35
  7. 2025Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Modelscited by 35
  8. 2021Contrastive Neural Architecture Search with Neural Architecture Comparatorscited by 34
  9. 2023Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?cited by 29
  10. 2026SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Passcited by 25
  11. 2022HyperTime: Implicit Neural Representation for Time Seriescited by 24
  12. 2022Transformers over Directed Acyclic Graphscited by 23
  13. 2022Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformercited by 22
  14. 2022Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysiscited by 22
  15. 2024FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Modelscited by 22
  16. 2023Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunitiescited by 21
  17. 2024HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Modelscited by 20
  18. 2024Functional Bilevel Optimization for Machine Learningcited by 20
  19. 2022Federated Learning with Heterogeneous Architectures using Graph HyperNetworkscited by 19
  20. 2023HyperINR: A Fast and Predictive Hypernetwork for Implicit Neural Representations via Knowledge Distillationcited by 19
  21. 2023Generating Behaviorally Diverse Policies with Latent Diffusion Modelscited by 19
  22. 2023Principled Weight Initialization for Hypernetworkscited by 19
  23. 2021HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworkscited by 18
  24. 2021Self-Regulation for Semantic Segmentationcited by 18
  25. 2021Towards Accurate and Compact Architectures via Neural Architecture Transformercited by 17
  26. 2021HyperSegNAS: Bridging One-Shot Neural Architecture Search with 3D Medical Image Segmentation using HyperNetcited by 14
  27. 2022NeRN -- Learning Neural Representations for Neural Networkscited by 14
  28. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  29. 2025Generative Modeling of Weights: Generalization or Memorization?cited by 14
  30. 2022CVAE-H: Conditionalizing Variational Autoencoders via Hypernetworks and Trajectory Forecasting for Autonomous Drivingcited by 13
  31. 2022Hyper-Representations for Pre-Training and Transfer Learningcited by 12
  32. 2023Focus Your Attention (with Adaptive IIR Filters)cited by 12
  33. 2021Regularization-Agnostic Compressed Sensing MRI Reconstruction with Hypernetworkscited by 9
  34. 2021Reconsidering CO2 emissions from Computer Visioncited by 9
  35. 2022Automated Graph Machine Learning: Approaches, Libraries, Benchmarks and Directionscited by 9
  36. 2022Combating Distribution Shift for Accurate Time Series Forecasting via Hypernetworkscited by 9
  37. 2022OCD: Learning to Overfit with Conditional Diffusion Modelscited by 9
  38. 2022Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammarscited by 9

50 most cited of 132 All 132 (JSON)