mlsanityData to 7 Oct 2026

Exploring Complicated Search Spaces with Interleaving-Free Sampling

arXiv 2112.02488 ↗First posted 5 Dec 2021cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The existing neural architecture search algorithms are mostly working on search spaces with short-distance connections. We argue that such designs, though safe and stable, obstacles the search algorithms from exploring more complicated scenarios. In this paper, we build the search algorithm upon a complicated search space with long-distance connections, and show that existing weight-sharing search algorithms mostly fail due to the existence of interleaved connections. Based on the observation, we present a simple yet effective algorithm named IF-NAS, where we perform a periodic sampling strategy to construct different sub-networks during the search procedure, avoiding the interleaved connections to emerge in any of them. In the proposed search space, IF-NAS outperform both random sampling and previous weight-sharing search algorithms by a significant margin. IF-NAS also generalizes to the micro cell-based spaces which are much easier. Our research emphasizes the importance of macro structure and we look forward to further efforts along this direction.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 55 references that are in the map. First posted on arXiv 5 Dec 2021.

Builds on

42 of 55 references.

References · 42

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2019Searching for MobileNetV3cited by 1,984
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  7. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  8. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  10. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  11. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  12. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
30 more
  1. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  2. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  3. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  4. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  5. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  6. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  7. 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
  8. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
  9. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 129
  10. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  11. 2020Can weight sharing outperform random architecture search? An investigation with TuNAScited by 68
  12. 2019BayesNAS: A Bayesian Approach for Neural Architecture Searchcited by 67
  13. 2019Densely Connected Search Space for More Flexible Neural Architecture Searchcited by 52
  14. 2019Understanding Architectures Learnt by Cell-based Neural Architecture Searchcited by 48
  15. 2019Customizable Architecture Search for Semantic Segmentationcited by 41
  16. 2020Hit-Detector: Hierarchical Trinity Architecture Search for Object Detectioncited by 32
  17. 2019ASAP: Architecture Search, Anneal and Prunecited by 29
  18. 2018You Only Search Once: Single Shot Neural Architecture Search via Direct Sparse Optimizationcited by 23
  19. 2020DOTS: Decoupling Operation and Topology in Differentiable Architecture Searchcited by 23
  20. 2020FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Searchcited by 16
  21. 2020GOLD-NAS: Gradual, One-Level, Differentiablecited by 15
  22. 2021Divergence-Regularized Multi-Agent Actor-Criticcited by 15
  23. 2020Discretization-Aware Architecture Searchcited by 14
  24. 2019Exploiting Operation Importance for Differentiable Neural Architecture Searchcited by 11
  25. 2021Semantic-Aware Generation for Self-Supervised Visual Representation Learningcited by 11
  26. 2021iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretizationcited by 10
  27. 2020Adaptive Linear Span Network for Object Skeleton Detectioncited by 9
  28. 2019Sub-Architecture Ensemble Pruning in Neural Architecture Searchcited by 7
  29. 2019SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Searchcited by 6
  30. 2019Hierarchical Neural Architecture Search via Operator Clusteringcited by 5

Built on by

Not cited yet.