mlsanityData to 7 Oct 2026

A Study on Encodings for Neural Architecture Search

arXiv 2007.04965 ↗First posted 9 Jul 2020cs.LG cs.NE stat.MLcited by 37

Not on the map. Why?

Abstract (arXiv)

Neural architecture search (NAS) has been extensively studied in the past few years. A popular approach is to represent each neural architecture in the search space as a directed acyclic graph (DAG), and then search over all DAGs by encoding the adjacency matrix and list of operations as a set of hyperparameters. Recent work has demonstrated that even small changes to the way each architecture is encoded can have a significant effect on the performance of NAS algorithms. In this work, we present the first formal study on the effect of architecture encodings for NAS, including a theoretical grounding and an empirical study. First we formally define architecture encodings and give a theoretical characterization on the scalability of the encodings we study Then we identify the main encoding-dependent subroutines which NAS algorithms employ, running experiments to show which encodings work best with each subroutine for many popular algorithms. The experiments act as an ablation study for prior work, disentangling the algorithmic and encoding-based contributions, as well as a guideline for future work. Our results demonstrate that NAS encodings are an important design decision which can have a significant impact on overall performance. Our code is available at https://github.com/naszilla/nas-encodings.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 10; 2023: 3; 2024: 7; 2025: 6; 2026: 3.

37 papers.

Sort:

By area:

  1. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  2. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88
  3. 2022NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easycited by 36
  4. 2022NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxiescited by 36
  5. 2021CATE: Computation-aware Neural Architecture Encoding with Transformerscited by 21
  6. 2021NAS-Bench-x11 and the Power of Learning Curvescited by 20
  7. 2022On Redundancy and Diversity in Cell-based Neural Architecture Searchcited by 18
  8. 2022Learning Where To Look -- Generative NAS is Surprisingly Efficientcited by 15
  9. 2022EZNAS: Evolving Zero Cost Proxies For Neural Architecture Scoringcited by 14
  10. 2022A Survey on Computationally Efficient Neural Architecture Searchcited by 12
  11. 2024Efficient Data Subset Selection to Generalize Training Across Models: Transductive and Inductive Networkscited by 12
  12. 2024Surprisingly Strong Performance Prediction with Neural Graph Featurescited by 11
25 more
  1. 2024Encodings for Prediction-based Neural Architecture Searchcited by 9
  2. 2024SeqNAS: Neural Architecture Search for Event Sequence Classificationcited by 8
  3. 2024On Latency Predictors for Neural Architecture Searchcited by 7
  4. 2021Approximate Neural Architecture Search via Operation Distribution Learningcited by 6
  5. 2023Multi-Predict: Few Shot Predictors For Efficient Neural Architecture Searchcited by 6
  6. 2021Graph-based Neural Architecture Search with Operation Embeddingscited by 5
  7. 2021MAPLE: Microprocessor A Priori for Latency Estimationcited by 5
  8. 2025On Accelerating Edge AI: Optimizing Resource-Constrained Environmentscited by 5
  9. 2021FEAR: A Simple Lightweight Method to Rank Architecturescited by 3
  10. 2021Mutation is all you needcited by 3
  11. 2022B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Searchcited by 3
  12. 2022Reducing Neural Architecture Search Spaces with Training-Free Statistics and Computational Graph Clusteringcited by 3
  13. 2022Tackling Neural Architecture Search With Quality Diversity Optimizationcited by 3
  14. 2022Visual Analysis of Neural Architecture Spaces for Summarizing Design Principlescited by 3
  15. 2023Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"cited by 3
  16. 2025Multi-View Encoders for Performance Prediction in LLM-Based Agentic Workflowscited by 3
  17. 2024Multi-Objective Evolutionary Neural Architecture Search for Recurrent Neural Networkscited by 2
  18. 2025Global optimization of graph acquisition functions for neural architecture searchcited by 2
  19. 2024SONATA: Self-adaptive Evolutionary Framework for Hardware-aware Neural Architecture Searchcited by 1
  20. 2025Learning to Flow from Generative Pretext Tasks for Neural Architecture Encodingcited by 1
  21. 2025Neural Architecture Search by Learning a Hierarchical Search Spacecited by 0
  22. 2025OptiProxy-NAS: Optimization Proxy based End-to-End Neural Architecture Searchcited by 0
  23. 2026Architecture Before the Formula: Individuating Neural Architecture Beyond the Composite Mapcited by 0
  24. 2026From Regression to Inference: Meta-Learning Predictors for Neural Architecture Searchcited by 0
  25. 2026Prediction-powered Neural Architecture Searchcited by 0