# DeepArchitect: Automatically Designing and Training Deep Architectures

arXiv: https://arxiv.org/abs/1704.08792 · HTML: https://mlsanity.com/paper/1704.08792/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2017.

## Built on by (most cited first, 28 of 28)

- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2103.12424/) (2021)
- [Neural Feature Search for RGB-Infrared Person Re-Identification](https://mlsanity.com/paper/2104.02366/) (2021)
- [NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy](https://mlsanity.com/paper/2201.13396/) (2022)
- [Prioritized Architecture Sampling with Monto-Carlo Tree Search](https://mlsanity.com/paper/2103.11922/) (2021)
- [CATE: Computation-aware Neural Architecture Encoding with Transformers](https://mlsanity.com/paper/2102.07108/) (2021)
- [NAS-Bench-x11 and the Power of Learning Curves](https://mlsanity.com/paper/2111.03602/) (2021)
- [Winning solutions and post-challenge analyses of the ChaLearn AutoDL challenge 2019](https://mlsanity.com/paper/2201.03801/) (2022)
- [Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift](https://mlsanity.com/paper/2108.09671/) (2021)
- [BM-NAS: Bilevel Multimodal Neural Architecture Search](https://mlsanity.com/paper/2104.09379/) (2021)
- [iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretization](https://mlsanity.com/paper/2108.11014/) (2021)
- [Neural Architecture Search via Combinatorial Multi-Armed Bandit](https://mlsanity.com/paper/2101.00336/) (2021)
- [Contrastive Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2102.10557/) (2021)
- [Multi-Objective Evolutionary Design of Composite Data-Driven Models](https://mlsanity.com/paper/2103.01301/) (2021)
- [DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions](https://mlsanity.com/paper/2403.01326/) (2024)
- [MARCO: Hardware-Aware Neural Architecture Search for Edge Devices with Multi-Agent Reinforcement Learning and Conformal Prediction Filtering](https://mlsanity.com/paper/2506.13755/) (2025)
- [Efficient Neural Architecture Search for Emotion Recognition](https://mlsanity.com/paper/2303.13653/) (2023)
- [Bilevel Optimization for Neural Architecture Search](https://mlsanity.com/paper/2606.29582/) (2026)
- [NAS-TC: Neural Architecture Search on Temporal Convolutions for Complex Action Recognition](https://mlsanity.com/paper/2104.01110/) (2021)
- [Activation Function Optimization Scheme for Image Classification](https://mlsanity.com/paper/2409.04915/) (2024)
- [Learning Morphisms with Gauss-Newton Approximation for Growing Networks](https://mlsanity.com/paper/2411.05855/) (2024)
- [PaRT: Parallel Learning Towards Robust and Transparent AI](https://mlsanity.com/paper/2201.09534/) (2022)
- [Local-to-Global Information Communication for Real-Time Semantic Segmentation Network Search](https://mlsanity.com/paper/2302.08481/) (2023)
- [Optimizing Neural Networks through Activation Function Discovery and Automatic Weight Initialization](https://mlsanity.com/paper/2304.03374/) (2023)
- [A Survey on Neural Architecture Search Based on Reinforcement Learning](https://mlsanity.com/paper/2409.18163/) (2024)
- [Neural Architecture Search by Learning a Hierarchical Search Space](https://mlsanity.com/paper/2503.21061/) (2025)
- [Searching Efficient Deep Architectures for Radar Target Detection using Monte-Carlo Tree Search](https://mlsanity.com/paper/2506.21772/) (2025)
- [confopt: A Library for Implementation and Evaluation of Gradient-based One-Shot NAS Methods](https://mlsanity.com/paper/2507.16533/) (2025)

## Abstract (arXiv)

In deep learning, performance is strongly affected by the choice of architecture and hyperparameters. While there has been extensive work on automatic hyperparameter optimization for simple spaces, complex spaces such as the space of deep architectures remain largely unexplored. As a result, the choice of architecture is done manually by the human expert through a slow trial and error process guided mainly by intuition. In this paper we describe a framework for automatically designing and training deep models. We propose an extensible and modular language that allows the human expert to compactly represent complex search spaces over architectures and their hyperparameters. The resulting search spaces are tree-structured and therefore easy to traverse. Models can be automatically compiled to computational graphs once values for all hyperparameters have been chosen. We can leverage the structure of the search space to introduce different model search algorithms, such as random search, Monte Carlo tree search (MCTS), and sequential model-based optimization (SMBO). We present experiments comparing the different algorithms on CIFAR-10 and show that MCTS and SMBO outperform random search. In addition, these experiments show that our framework can be used effectively for model discovery, as it is possible to describe expressive search spaces and discover competitive models without much effort from the human expert. Code for our framework and experiments has been made publicly available.