# Hierarchical Representations for Efficient Architecture Search

arXiv: https://arxiv.org/abs/1711.00436 · HTML: https://mlsanity.com/paper/1711.00436/

## Key facts

- Cited by 191 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2017.

## Built on by (most cited first, 30 of 191)

- [Primer: Searching for Efficient Transformers for Language Modeling](https://mlsanity.com/paper/2109.08668/) (2021)
- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [Rethinking Architecture Selection in Differentiable NAS](https://mlsanity.com/paper/2108.04392/) (2021)
- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients](https://mlsanity.com/paper/2301.11300/) (2023)
- [CktGNN: Circuit Graph Neural Network for Electronic Design Automation](https://mlsanity.com/paper/2308.16406/) (2023)
- [Neural Feature Search for RGB-Infrared Person Re-Identification](https://mlsanity.com/paper/2104.02366/) (2021)
- [Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations](https://mlsanity.com/paper/2205.13571/) (2022)
- [Global2Local: Efficient Structure Search for Video Action Segmentation](https://mlsanity.com/paper/2101.00910/) (2021)
- [Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overview](https://mlsanity.com/paper/2206.07438/) (2022)
- [Contrastive Neural Architecture Search with Neural Architecture Comparators](https://mlsanity.com/paper/2103.05471/) (2021)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [Towards Execution-Grounded Automated AI Research](https://mlsanity.com/paper/2601.14525/) (2026)
- [Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Search](https://mlsanity.com/paper/2101.11342/) (2021)
- [Rethinking Graph Neural Architecture Search from Message-passing](https://mlsanity.com/paper/2103.14282/) (2021)
- [Core-set Sampling for Efficient Neural Architecture Search](https://mlsanity.com/paper/2107.06869/) (2021)
- [ChatGPT as your Personal Data Scientist](https://mlsanity.com/paper/2305.13657/) (2023)
- [CATE: Computation-aware Neural Architecture Encoding with Transformers](https://mlsanity.com/paper/2102.07108/) (2021)
- [PSViT: Better Vision Transformer via Token Pooling and Attention Sharing](https://mlsanity.com/paper/2108.03428/) (2021)
- [Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2307.01998/) (2023)
- [DPNAS: Neural Architecture Search for Deep Learning with Differential Privacy](https://mlsanity.com/paper/2110.08557/) (2021)
- [PACE: A Parallelizable Computation Encoder for Directed Acyclic Graphs](https://mlsanity.com/paper/2203.10304/) (2022)
- [Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation](https://mlsanity.com/paper/2205.01271/) (2022)
- [MSINet: Twins Contrastive Search of Multi-Scale Interaction for Object ReID](https://mlsanity.com/paper/2303.07065/) (2023)
- [TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs](https://mlsanity.com/paper/2308.13490/) (2023)
- [Sphynx: ReLU-Efficient Network Design for Private Inference](https://mlsanity.com/paper/2106.11755/) (2021)
- [Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2106.11890/) (2021)
- [Generalization Properties of NAS under Activation and Skip Connection Search](https://mlsanity.com/paper/2209.07238/) (2022)
- [Computation-efficient Deep Learning for Computer Vision: A Survey](https://mlsanity.com/paper/2308.13998/) (2023)

## Abstract (arXiv)

We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance. Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human experts, and an expressive search space that supports complex topologies. Our algorithm efficiently discovers architectures that outperform a large number of manually designed models for image classification, obtaining top-1 error of 3.6% on CIFAR-10 and 20.3% when transferred to ImageNet, which is competitive with the best existing neural architecture search approaches. We also present results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10 and 0.1% less on ImageNet whilst reducing the search time from 36 hours down to 1 hour.