# Learnable Embedding Space for Efficient Neural Architecture Compression

arXiv: https://arxiv.org/abs/1902.00383 · HTML: https://mlsanity.com/paper/1902.00383/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2019.

## Built on by (most cited first, 10 of 10)

- [Towards Accurate and Compact Architectures via Neural Architecture Transformer](https://mlsanity.com/paper/2102.10301/) (2021)
- [HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration](https://mlsanity.com/paper/2212.04359/) (2022)
- [Neighborhood-Aware Neural Architecture Search](https://mlsanity.com/paper/2105.06369/) (2021)
- [Combining Multi-Objective Bayesian Optimization with Reinforcement Learning for TinyML](https://mlsanity.com/paper/2305.14109/) (2023)
- [Enhanced Gradient for Differentiable Architecture Search](https://mlsanity.com/paper/2103.12529/) (2021)
- [Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation](https://mlsanity.com/paper/2110.09674/) (2021)
- [Robustifying and Boosting Training-Free Neural Architecture Search](https://mlsanity.com/paper/2403.07591/) (2024)
- [Streaming Self-Training via Domain-Agnostic Unlabeled Images](https://mlsanity.com/paper/2104.03309/) (2021)
- [CWP: Instance complexity weighted channel-wise soft masks for network pruning](https://mlsanity.com/paper/2209.03534/) (2022)
- [Discovering Sparse Recovery Algorithms Using Neural Architecture Search](https://mlsanity.com/paper/2512.21563/) (2025)

## Abstract (arXiv)

We propose a method to incrementally learn an embedding space over the domain of network architectures, to enable the careful selection of architectures for evaluation during compressed architecture search. Given a teacher network, we search for a compressed network architecture by using Bayesian Optimization (BO) with a kernel function defined over our proposed embedding space to select architectures for evaluation. We demonstrate that our search algorithm can significantly outperform various baseline methods, such as random search and reinforcement learning (Ashok et al., 2018). The compressed architectures found by our method are also better than the state-of-the-art manually-designed compact architecture ShuffleNet (Zhang et al., 2018). We also demonstrate that the learned embedding space can be transferred to new settings for architecture search, such as a larger teacher network or a teacher network in a different architecture family, without any training. Code is publicly available here: https://github.com/Friedrich1006/ESNAC .