# Neural Network Architecture Optimization through Submodularity and Supermodularity

arXiv: https://arxiv.org/abs/1609.00074 · HTML: https://mlsanity.com/paper/1609.00074/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2016.

## Built on by (most cited first, 3 of 3)

- [DECORE: Deep Compression with Reinforcement Learning](https://mlsanity.com/paper/2106.06091/) (2021)
- [Text Analysis Using Deep Neural Networks in Digital Humanities and Information Science](https://mlsanity.com/paper/2307.16217/) (2023)
- [Toward a Period-Specific Optimized Neural Network for OCR Error Correction of Historical Hebrew Texts](https://mlsanity.com/paper/2307.16213/) (2023)

## Abstract (arXiv)

Deep learning models' architectures, including depth and width, are key factors influencing models' performance, such as test accuracy and computation time. This paper solves two problems: given computation time budget, choose an architecture to maximize accuracy, and given accuracy requirement, choose an architecture to minimize computation time. We convert this architecture optimization into a subset selection problem. With accuracy's submodularity and computation time's supermodularity, we propose efficient greedy optimization algorithms. The experiments demonstrate our algorithm's ability to find more accurate models or faster models. By analyzing architecture evolution with growing time budget, we discuss relationships among accuracy, time and architecture, and give suggestions on neural network architecture design.