mlsanityData to 14 Oct 2026

Neural Architecture Search by Learning a Hierarchical Search Space

arXiv 2503.21061 ↗First posted 27 Mar 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Monte-Carlo Tree Search (MCTS) is a powerful tool for many non-differentiable search related problems such as adversarial games. However, the performance of such approach highly depends on the order of the nodes that are considered at each branching of the tree. If the first branches cannot distinguish between promising and deceiving configurations for the final task, the efficiency of the search is exponentially reduced. In Neural Architecture Search (NAS), as only the final architecture matters, the visiting order of the branching can be optimized to improve learning. In this paper, we study the application of MCTS to NAS for image classification. We analyze several sampling methods and branching alternatives for MCTS and propose to learn the branching by hierarchical clustering of architectures based on their similarity. The similarity is measured by the pairwise distance of output vectors of architectures. Extensive experiments on two challenging benchmarks on CIFAR10 and ImageNet show that MCTS, if provided with a good branching hierarchy, can yield promising solutions more efficiently than other approaches for NAS problems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 56 of its 67 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

56 of 67 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  4. 2017Squeeze-and-Excitation Networkscited by 4,789
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  8. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  10. 2017Progressive Neural Architecture Searchcited by 487
  11. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  12. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
38 more
  1. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  2. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  3. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 260
  4. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 254
  5. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 254
  6. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 179
  7. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  8. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  9. 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 130
  10. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  11. 2019D-VAE: A Variational Autoencoder for Directed Acyclic Graphscited by 98
  12. 2020NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Sizecited by 98
  13. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  14. 2019Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hourscited by 87
  15. 2017Boltzmann Exploration Done Rightcited by 69
  16. 2019SGAS: Sequential Greedy Architecture Searchcited by 69
  17. 2020GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernetcited by 69
  18. 2022β-DARTS: Beta-Decay Regularization for Differentiable Architecture Searchcited by 64
  19. 2020Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?cited by 59
  20. 2017Evolving Deep Convolutional Neural Networks for Image Classificationcited by 55
  21. 2020A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAScited by 55
  22. 2021GLiT: Neural Architecture Search for Global and Local Image Transformercited by 53
  23. 2020Few-shot Neural Architecture Searchcited by 50
  24. 2023FFCV: Accelerating Training by Removing Data Bottleneckscited by 49
  25. 2018Automatically designing CNN architectures using genetic algorithm for image classificationcited by 48
  26. 2019Progressive DARTS: Bridging the Optimization Gap for NAS in the Wildcited by 40
  27. 2020A Study on Encodings for Neural Architecture Searchcited by 38
  28. 2019AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Searchcited by 33
  29. 2021Prioritized Architecture Sampling with Monto-Carlo Tree Searchcited by 32
  30. 2017DeepArchitect: Automatically Designing and Training Deep Architecturescited by 28
  31. 2022Generalizing Few-Shot NAS with Gradient Matchingcited by 25
  32. 2022Shapley-NAS: Discovering Operation Contribution for Neural Architecture Searchcited by 23
  33. 2021K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernetscited by 22
  34. 2021CATE: Computation-aware Neural Architecture Encoding with Transformerscited by 21
  35. 2022Learning Where To Look -- Generative NAS is Surprisingly Efficientcited by 15
  36. 2020Smooth Variational Graph Embeddings for Efficient Neural Architecture Searchcited by 12
  37. 2024Monte Carlo Tree Search with Boltzmann Explorationcited by 12
  38. 2020Neural Architecture Optimization with Graph VAEcited by 9

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.