# Neural Architecture Optimization with Graph VAE

arXiv: https://arxiv.org/abs/2006.10310 · HTML: https://mlsanity.com/paper/2006.10310/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 9 of 9)

- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization](https://mlsanity.com/paper/2404.16795/) (2024)
- [Graph-based Neural Architecture Search with Operation Embeddings](https://mlsanity.com/paper/2105.04885/) (2021)
- [Simultaneous Weight and Architecture Optimization for Neural Networks](https://mlsanity.com/paper/2410.08339/) (2024)
- [Automating Neural Architecture Design without Search](https://mlsanity.com/paper/2204.11838/) (2022)
- [Neural Architecture Search by Learning a Hierarchical Search Space](https://mlsanity.com/paper/2503.21061/) (2025)
- [SWAT-NN: Simultaneous Weights and Architecture Training for Neural Networks in a Latent Space](https://mlsanity.com/paper/2506.08270/) (2025)
- [A Multi-stage Constrained Optimization Framework for Data-driven Problems](https://mlsanity.com/paper/2607.23480/) (2026)

## Abstract (arXiv)

Due to their high computational efficiency on a continuous space, gradient optimization methods have shown great potential in the neural architecture search (NAS) domain. The mapping of network representation from the discrete space to a latent space is the key to discovering novel architectures, however, existing gradient-based methods fail to fully characterize the networks. In this paper, we propose an efficient NAS approach to optimize network architectures in a continuous space, where the latent space is built upon variational autoencoder (VAE) and graph neural networks (GNN). The framework jointly learns four components: the encoder, the performance predictor, the complexity predictor and the decoder in an end-to-end manner. The encoder and the decoder belong to a graph VAE, mapping architectures between continuous representations and network architectures. The predictors are two regression models, fitting the performance and computational complexity, respectively. Those predictors ensure the discovered architectures characterize both excellent performance and high computational efficiency. Extensive experiments demonstrate our framework not only generates appropriate continuous representations but also discovers powerful neural architectures.