# Smooth Variational Graph Embeddings for Efficient Neural Architecture Search

arXiv: https://arxiv.org/abs/2010.04683 · HTML: https://mlsanity.com/paper/2010.04683/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2020.

## Built on by (most cited first, 12 of 12)

- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [CATE: Computation-aware Neural Architecture Encoding with Transformers](https://mlsanity.com/paper/2102.07108/) (2021)
- [Learning Where To Look -- Generative NAS is Surprisingly Efficient](https://mlsanity.com/paper/2203.08734/) (2022)
- [Neural Architecture Design and Robustness: A Dataset](https://mlsanity.com/paper/2306.06712/) (2023)
- [Multi-conditioned Graph Diffusion for Neural Architecture Search](https://mlsanity.com/paper/2403.06020/) (2024)
- [Efficient Data Subset Selection to Generalize Training Across Models: Transductive and Inductive Networks](https://mlsanity.com/paper/2409.12255/) (2024)
- [Regression Language Models for Code](https://mlsanity.com/paper/2509.26476/) (2025)
- [Is Differentiable Architecture Search truly a One-Shot Method?](https://mlsanity.com/paper/2108.05647/) (2021)
- [Accuracy Prediction for NAS Acceleration using Feature Selection and Extrapolation](https://mlsanity.com/paper/2211.12419/) (2022)
- [FR-NAS: Forward-and-Reverse Graph Predictor for Efficient Neural Architecture Search](https://mlsanity.com/paper/2404.15622/) (2024)
- [Neural Architecture Search by Learning a Hierarchical Search Space](https://mlsanity.com/paper/2503.21061/) (2025)
- [Arch-VQ: Discrete Architecture Representation Learning with Autoregressive Priors](https://mlsanity.com/paper/2503.22063/) (2025)

## Abstract (arXiv)

Neural architecture search (NAS) has recently been addressed from various directions, including discrete, sampling-based methods and efficient differentiable approaches. While the former are notoriously expensive, the latter suffer from imposing strong constraints on the search space. Architecture optimization from a learned embedding space for example through graph neural network based variational autoencoders builds a middle ground and leverages advantages from both sides. Such approaches have recently shown good performance on several benchmarks. Yet, their stability and predictive power heavily depends on their capacity to reconstruct networks from the embedding space. In this paper, we propose a two-sided variational graph autoencoder, which allows to smoothly encode and accurately reconstruct neural architectures from various search spaces. We evaluate the proposed approach on neural architectures defined by the ENAS approach, the NAS-Bench-101 and the NAS-Bench-201 search space and show that our smooth embedding space allows to directly extrapolate the performance prediction to architectures outside the seen domain (e.g. with more operations). Thus, it facilitates to predict good network architectures even without expensive Bayesian optimization or reinforcement learning.