mlsanityData to 13 Oct 2026

SMASH: One-Shot Model Architecture Search through HyperNetworks

arXiv 1708.05344 ↗First posted 17 Aug 2017cs.LGcited by 201

Not on the map. Why?

Abstract (arXiv)

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at https://github.com/ajbrock/SMASH

Key facts. Cited by 201 papers in the map (data through 13 Oct 2026). First posted on arXiv 17 Aug 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 62; 2022: 48; 2023: 38; 2024: 24; 2025: 19; 2026: 9.

201 papers.

Sort:

By area:

  1. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 187
  2. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  3. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 143
  4. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  5. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  6. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  7. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  8. 2021HyperMorph: Amortized Hyperparameter Learning for Image Registrationcited by 85
  9. 2021Dynamic Slimmable Networkcited by 83
  10. 2023Sensitivity-Aware Visual Parameter-Efficient Fine-Tuningcited by 74
  11. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  12. 2022Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learningcited by 57
38 more
  1. 2021Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusioncited by 56
  2. 2022Neural Architecture Search for Spiking Neural Networkscited by 53
  3. 2021Searching the Search Space of Vision Transformercited by 40
  4. 2024Neural Network Diffusioncited by 40
  5. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 39
  6. 2022Automated Progressive Learning for Efficient Training of Vision Transformerscited by 36
  7. 2022Learning the Effect of Registration Hyperparameters with HyperMorphcited by 35
  8. 2022DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalizationcited by 35
  9. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 35
  10. 2021Neural Architecture Search with Random Labelscited by 33
  11. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 33
  12. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  13. 2021AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Modelscited by 31
  14. 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
  15. 2023Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Modelscited by 30
  16. 2023Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?cited by 29
  17. 2024Multilinear Mixture of Experts: Scalable Expert Specialization through Factorizationcited by 28
  18. 2021Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Searchcited by 27
  19. 2021CompOFA: Compound Once-For-All Networks for Faster Multi-Platform Deploymentcited by 26
  20. 2021PP-PicoDet: A Better Real-Time Object Detector on Mobile Devicescited by 25
  21. 2021EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Searchcited by 24
  22. 2022HyperTime: Implicit Neural Representation for Time Seriescited by 24
  23. 2021Core-set Sampling for Efficient Neural Architecture Searchcited by 23
  24. 2022Controllable Dynamic Multi-Task Architecturescited by 21
  25. 2021HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Searchcited by 20
  26. 2021Analyzing and Mitigating Interference in Neural Architecture Searchcited by 20
  27. 2022Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolutioncited by 20
  28. 2024HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Modelscited by 20
  29. 2024Functional Bilevel Optimization for Machine Learningcited by 20
  30. 2021OPANAS: One-Shot Path Aggregation Network Architecture Search for Object Detectioncited by 19
  31. 2022Federated Learning with Heterogeneous Architectures using Graph HyperNetworkscited by 19
  32. 2023Principled Weight Initialization for Hypernetworkscited by 19
  33. 2021HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworkscited by 18
  34. 2022Carbon Footprint of Selecting and Training Deep Learning Models for Medical Image Analysiscited by 18
  35. 2022Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training?cited by 18
  36. 2025Pareto Set Learning for Multi-Objective Reinforcement Learningcited by 18
  37. 2021Towards Accurate and Compact Architectures via Neural Architecture Transformercited by 17
  38. 2024Condition-Aware Neural Network for Controlled Image Generationcited by 16

50 most cited of 201 All 201 (JSON)