mlsanityData to 7 Oct 2026

Conditional super-network weights

arXiv 2104.11522 ↗First posted 23 Apr 2021cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern Neural Architecture Search methods have repeatedly broken state-of-the-art results for several disciplines. The super-network, a central component of many such methods, enables quick estimates of accuracy or loss statistics for any architecture in the search space. They incorporate the network weights of all candidate architectures and can thus approximate specific ones by applying the respective operations. However, this design ignores potential dependencies between consecutive operations. We extend super-networks with conditional weights that depend on combinations of choices and analyze their effect. Experiments in NAS-Bench 201 and NAS-Bench-Macro-based search spaces show improvements in the architecture selection and that the resource overhead is nearly negligible for sequential network designs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 23 Apr 2021.

Builds on

25 of 39 references.

References · 25

  1. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  2. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  3. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  4. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  5. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  6. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  7. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  8. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  9. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  10. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  11. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  12. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
13 more
  1. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  2. 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 129
  3. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 129
  4. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  5. 2019Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hourscited by 87
  6. 2019Blockwisely Supervised Neural Architecture Search with Knowledge Distillationcited by 81
  7. 2020AttentiveNAS: Improving Neural Architecture Search via Attentive Samplingcited by 65
  8. 2020Few-shot Neural Architecture Searchcited by 50
  9. 2020Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Searchcited by 38
  10. 2021Prioritized Architecture Sampling with Monto-Carlo Tree Searchcited by 32
  11. 2020Deeper Insights into Weight Sharing in Neural Architecture Searchcited by 24
  12. 2021HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Searchcited by 20
  13. 2020How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAScited by 17

Built on by

Not cited yet.