mlsanityData to 7 Oct 2026

Understanding Architectures Learnt by Cell-based Neural Architecture Search

arXiv 1909.09569 ↗First posted 20 Sep 2019cs.LG cs.CV stat.MLcited by 48

Not on the map. Why?

Abstract (arXiv)

Neural architecture search (NAS) searches architectures automatically for given tasks, e.g., image classification and language modeling. Improving the search efficiency and effectiveness have attracted increasing attention in recent years. However, few efforts have been devoted to understanding the generated architectures. In this paper, we first reveal that existing NAS algorithms (e.g., DARTS, ENAS) tend to favor architectures with wide and shallow cell structures. These favorable architectures consistently achieve fast convergence and are consequently selected by NAS algorithms. Our empirical and theoretical study further confirms that their fast convergence derives from their smooth loss landscape and accurate gradient information. Nonetheless, these architectures may not necessarily lead to better generalization performance compared with other candidate architectures in the same search space, and therefore further improvement is possible by revising existing NAS algorithms.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 14; 2023: 6; 2024: 4; 2025: 1; 2026: 1.

48 papers.

Sort:

  1. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  2. 2021A Survey on Green Deep Learningcited by 52
  3. 2022Neural Architecture Search for Spiking Neural Networkscited by 52
  4. 2021Neural Architecture Search with Random Labelscited by 33
  5. 2021AdvRush: Searching for Adversarially Robust Neural Architecturescited by 28
  6. 2021NASI: Label- and Data-agnostic Neural Architecture Search at Initializationcited by 28
  7. 2022Sample-Then-Optimize Batch Neural Thompson Samplingcited by 25
  8. 2021Analyzing and Mitigating Interference in Neural Architecture Searchcited by 20
  9. 2023HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networkscited by 20
  10. 2022On Redundancy and Diversity in Cell-based Neural Architecture Searchcited by 18
  11. 2021Towards Accurate and Compact Architectures via Neural Architecture Transformercited by 17
  12. 2022Unifying and Boosting Gradient-Based Training-Free Neural Architecture Searchcited by 17
36 more
  1. 2022Automated Quantum Circuit Design with Nested Monte Carlo Tree Searchcited by 16
  2. 2022Generalization Properties of NAS under Activation and Skip Connection Searchcited by 14
  3. 2024Flexora: Flexible Low Rank Adaptation for Large Language Modelscited by 13
  4. 2021Learning Diverse-Structured Networks for Adversarial Robustnesscited by 12
  5. 2024Surprisingly Strong Performance Prediction with Neural Graph Featurescited by 11
  6. 2021Zero-Cost Operation Scoring in Differentiable Architecture Searchcited by 9
  7. 2021RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halvingcited by 9
  8. 2021Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metricscited by 7
  9. 2021On the Security Risks of AutoMLcited by 7
  10. 2021L2NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learningcited by 6
  11. 2023An algorithmic framework for the optimization of deep neural networks architectures and hyperparameterscited by 6
  12. 2021Neural Architecture Search via Combinatorial Multi-Armed Banditcited by 5
  13. 2021BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rulecited by 5
  14. 2022Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentationcited by 5
  15. 2022On the Privacy Risks of Cell-Based NAS Architecturescited by 5
  16. 2022Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysiscited by 4
  17. 2024Robustifying and Boosting Training-Free Neural Architecture Searchcited by 4
  18. 2024OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strengthcited by 4
  19. 2021FEAR: A Simple Lightweight Method to Rank Architecturescited by 3
  20. 2021Guided Evolution for Neural Architecture Searchcited by 3
  21. 2023FTSO: Effective NAS via First Topology Second Operatorcited by 3
  22. 2023GeNAS: Neural Architecture Search with Better Generalizationcited by 3
  23. 2022A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Searchcited by 2
  24. 2022Towards Self-supervised and Weight-preserving Neural Architecture Searchcited by 2
  25. 2022Guided Evolutionary Neural Architecture Search With Efficient Performance Estimationcited by 2
  26. 2023Rethink DARTS Search Space and Renovate a New Benchmarkcited by 2
  27. 2025Architecture-Aware Minimization (A2M): How to Find Flat Minima in Neural Architecture Searchcited by 2
  28. 2021AutoHEnsGNN: Winning Solution to AutoGraph Challenge for KDD Cup 2020cited by 1
  29. 2022Meta-Learning of NAS for Few-shot Learning in Medical Image Applicationscited by 1
  30. 2022α DARTS Once More: Enhancing Differentiable Architecture Search by Masked Image Modelingcited by 1
  31. 2023Small Temperature is All You Need for Differentiable Architecture Searchcited by 1
  32. 2021MS-DARTS: Mean-Shift Based Differentiable Architecture Searchcited by 0
  33. 2021Designing the Architecture of a Convolutional Neural Network Automatically for Diabetic Retinopathy Diagnosiscited by 0
  34. 2021Exploring Complicated Search Spaces with Interleaving-Free Samplingcited by 0
  35. 2021DARTS without a Validation Set: Optimizing the Marginal Likelihoodcited by 0
  36. 2026HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Searchcited by 0