mlsanityData to 7 Oct 2026

Towards Less Constrained Macro-Neural Architecture Search

arXiv 2203.05508 ↗First posted 10 Mar 2022cs.CV cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Networks found with Neural Architecture Search (NAS) achieve state-of-the-art performance in a variety of tasks, out-performing human-designed networks. However, most NAS methods heavily rely on human-defined assumptions that constrain the search: architecture's outer-skeletons, number of layers, parameter heuristics and search spaces. Additionally, common search spaces consist of repeatable modules (cells) instead of fully exploring the architecture's search space by designing entire architectures (macro-search). Imposing such constraints requires deep human expertise and restricts the search to pre-defined settings. In this paper, we propose LCMNAS, a method that pushes NAS to less constrained search spaces by performing macro-search without relying on pre-defined heuristics or bounded search spaces. LCMNAS introduces three components for the NAS pipeline: i) a method that leverages information about well-known architectures to autonomously generate complex search spaces based on Weighted Directed Graphs with hidden properties, ii) an evolutionary search strategy that generates complete architectures from scratch, and iii) a mixed-performance estimation approach that combines information about architectures at initialization stage and lower fidelity estimates to infer their trainability and capacity to model complex functions. We present experiments in 13 different data sets showing that LCMNAS is capable of generating both cell and macro-based architectures with minimal GPU computation and state-of-the-art results. More, we conduct extensive studies on the importance of different NAS components in both cell and macro-based settings. Code for reproducibility is public at https://github.com/VascoLopes/LCMNAS.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 67 of its 89 references that are in the map. First posted on arXiv 10 Mar 2022.

Builds on

67 of 89 references.

References · 67

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2016Wide Residual Networkscited by 3,057
  9. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  10. 2019Searching for MobileNetV3cited by 1,984
  11. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  12. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
38 more
  1. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  2. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  4. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  5. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  6. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  7. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  8. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  9. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  10. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  11. 2017Progressive Neural Architecture Searchcited by 483
  12. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  13. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  14. 2017Large-Scale Evolution of Image Classifierscited by 348
  15. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  16. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  17. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  18. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  19. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  20. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  21. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
  22. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  23. 2020Neural Architecture Search without Trainingcited by 241
  24. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  25. 2016FractalNet: Ultra-Deep Neural Networks without Residualscited by 216
  26. 2019A Survey of the Recent Architectures of Deep Convolutional Neural Networkscited by 211
  27. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  28. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
  29. 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
  30. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
  31. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  32. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  33. 2018Efficient Multi-objective Neural Architecture Search via Lamarckian Evolutioncited by 151
  34. 2020A Survey on Evolutionary Neural Architecture Searchcited by 142
  35. 2019AutoGAN: Neural Architecture Search for Generative Adversarial Networkscited by 94
  36. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88
  37. 2019DetNAS: Backbone Search for Object Detectioncited by 80
  38. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 0; 2025: 1.

4 papers.

  1. 2023Are Neural Architecture Search Benchmarks Well Designed? A Deeper Look Into Operation Importancecited by 3
  2. 2022Guided Evolutionary Neural Architecture Search With Efficient Performance Estimationcited by 2
  3. 2023Balanced Mixture of SuperNets for Learning the CNN Pooling Architecturecited by 1
  4. 2025Efficient Global Neural Architecture Searchcited by 0