mlsanityData to 14 Oct 2026

A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Search

arXiv 2204.03916 ↗First posted 8 Apr 2022cs.CV cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

This survey focuses on categorizing and evaluating the methods of supernet optimization in the field of Neural Architecture Search (NAS). Supernet optimization involves training a single, over-parameterized network that encompasses the search space of all possible network architectures. The survey analyses supernet optimization methods based on their approaches to spatial and temporal optimization. Spatial optimization relates to optimizing the architecture and parameters of the supernet and its subnets, while temporal optimization deals with improving the efficiency of selecting architectures from the supernet. The benefits, limitations, and potential applications of these methods in various tasks and settings, including transferability, domain generalization, and Transformer models, are also discussed.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 43 of its 52 references that are in the map. First posted on arXiv 8 Apr 2022.

Builds on

43 of 52 references.

References · 43

  1. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  2. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  4. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  5. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  6. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  7. 2017Large-Scale Evolution of Image Classifierscited by 348
  8. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  9. 2020BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Modelscited by 188
  10. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 188
  11. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  12. 2020AutoML-Zero: Evolving Machine Learning Algorithms From Scratchcited by 143
31 more
  1. 2020HAT: Hardware-Aware Transformers for Efficient Natural Language Processingcited by 136
  2. 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 130
  3. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  4. 2019Blockwisely Supervised Neural Architecture Search with Knowledge Distillationcited by 81
  5. 2019SGAS: Sequential Greedy Architecture Searchcited by 69
  6. 2020GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernetcited by 69
  7. 2020AttentiveNAS: Improving Neural Architecture Search via Attentive Samplingcited by 66
  8. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  9. 2019Densely Connected Search Space for More Flexible Neural Architecture Searchcited by 52
  10. 2020Few-shot Neural Architecture Searchcited by 50
  11. 2019Understanding Architectures Learnt by Cell-based Neural Architecture Searchcited by 48
  12. 2019AtomNAS: Fine-Grained End-to-End Neural Architecture Searchcited by 40
  13. 2021HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformerscited by 39
  14. 2020Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Searchcited by 38
  15. 2021AlphaNet: Improved Training of Supernets with Alpha-Divergencecited by 36
  16. 2020Distilling Optimal Neural Networks: Rapid Search in Diverse Spacescited by 32
  17. 2021ViTAS: Vision Transformer Architecture Searchcited by 31
  18. 2021NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalizationcited by 28
  19. 2021Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Searchcited by 27
  20. 2022Generalizing Few-Shot NAS with Gradient Matchingcited by 25
  21. 2020DOTS: Decoupling Operation and Topology in Differentiable Architecture Searchcited by 23
  22. 2021Analyzing and Mitigating Interference in Neural Architecture Searchcited by 20
  23. 2020Cyclic Differentiable Architecture Searchcited by 18
  24. 2021FBNetV5: Neural Architecture Search for Multiple Tasks in One Runcited by 15
  25. 2020DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Searchcited by 14
  26. 2021SPIDER: Searching Personalized Neural Architecture for Federated Learningcited by 13
  27. 2020Progressive Automatic Design of Search Space for One-Shot Neural Architecture Searchcited by 10
  28. 2020Neural Architecture Search as Sparse Supernetcited by 10
  29. 2020Network Architecture Search for Domain Adaptationcited by 9
  30. 2020MixPath: A Unified Approach for One-shot Neural Architecture Searchcited by 7
  31. 2021ESAI: Efficient Split Artificial Intelligence via Early Exiting Using Neural Architecture Searchcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Artificial Bee Colony optimization of Deep Convolutional Neural Networks in the context of Biomedical Imagingcited by 0
  2. 2025DPFNAS: Differential Privacy-Enhanced Federated Neural Architecture Search for 6G Edge Intelligencecited by 0