# Towards modular and programmable architecture search

arXiv: https://arxiv.org/abs/1909.13404 · HTML: https://mlsanity.com/paper/1909.13404/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2019.

## Built on by (most cited first, 11 of 11)

- [Modular Deep Learning](https://mlsanity.com/paper/2302.11529/) (2023)
- [A survey on multi-objective hyperparameter optimization algorithms for Machine Learning](https://mlsanity.com/paper/2111.13755/) (2021)
- [PyGlove: Symbolic Programming for Automated Machine Learning](https://mlsanity.com/paper/2101.08809/) (2021)
- [Personalized Algorithm Generation: A Case Study in Learning ODE Integrators](https://mlsanity.com/paper/2105.01303/) (2021)
- [Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars](https://mlsanity.com/paper/2211.01842/) (2022)
- [A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms](https://mlsanity.com/paper/2211.12386/) (2022)
- [DP-HyPO: An Adaptive Private Hyperparameter Optimization Framework](https://mlsanity.com/paper/2306.05734/) (2023)
- [Searching for More Efficient Dynamic Programs](https://mlsanity.com/paper/2109.06966/) (2021)
- [DREAM: Debugging and Repairing AutoML Pipelines](https://mlsanity.com/paper/2401.00379/) (2023)
- [confopt: A Library for Implementation and Evaluation of Gradient-based One-Shot NAS Methods](https://mlsanity.com/paper/2507.16533/) (2025)
- [Neural Guided Sampling for Quantum Circuit Optimization](https://mlsanity.com/paper/2510.12430/) (2025)

## Abstract (arXiv)

Neural architecture search methods are able to find high performance deep learning architectures with minimal effort from an expert. However, current systems focus on specific use-cases (e.g. convolutional image classifiers and recurrent language models), making them unsuitable for general use-cases that an expert might wish to write. Hyperparameter optimization systems are general-purpose but lack the constructs needed for easy application to architecture search. In this work, we propose a formal language for encoding search spaces over general computational graphs. The language constructs allow us to write modular, composable, and reusable search space encodings and to reason about search space design. We use our language to encode search spaces from the architecture search literature. The language allows us to decouple the implementations of the search space and the search algorithm, allowing us to expose search spaces to search algorithms through a consistent interface. Our experiments show the ease with which we can experiment with different combinations of search spaces and search algorithms without having to implement each combination from scratch. We release an implementation of our language with this paper.