# ModuleNet: Knowledge-inherited Neural Architecture Search

arXiv: https://arxiv.org/abs/2004.05020 · HTML: https://mlsanity.com/paper/2004.05020/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2020.

## Built on by (most cited first, 11 of 11)

- [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022)
- [BViT: Broad Attention based Vision Transformer](https://mlsanity.com/paper/2202.06268/) (2022)
- [ABCP: Automatic Block-wise and Channel-wise Network Pruning via Joint Search](https://mlsanity.com/paper/2110.03858/) (2021)
- [OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strength](https://mlsanity.com/paper/2409.14433/) (2024)
- [Stacked BNAS: Rethinking Broad Convolutional Neural Network for Neural Architecture Search](https://mlsanity.com/paper/2111.07722/) (2021)
- [Vision Transformer with Convolutions Architecture Search](https://mlsanity.com/paper/2203.10435/) (2022)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [Towards Neural Architecture Search for Transfer Learning in 6G Networks](https://mlsanity.com/paper/2406.02333/) (2024)
- [A formal definition and meta-model for a machine theory of mind](https://mlsanity.com/paper/2606.03471/) (2026)
- [Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity](https://mlsanity.com/paper/2207.00751/) (2022)
- [Multi-modal Learning based Prediction for Disease](https://mlsanity.com/paper/2307.09823/) (2023)

## Abstract (arXiv)

Although Neural Architecture Search (NAS) can bring improvement to deep models, they always neglect precious knowledge of existing models. The computation and time costing property in NAS also means that we should not start from scratch to search, but make every attempt to reuse the existing knowledge. In this paper, we discuss what kind of knowledge in a model can and should be used for new architecture design. Then, we propose a new NAS algorithm, namely ModuleNet, which can fully inherit knowledge from existing convolutional neural networks. To make full use of existing models, we decompose existing models into different textitmodules which also keep their weights, consisting of a knowledge base. Then we sample and search for new architecture according to the knowledge base. Unlike previous search algorithms, and benefiting from inherited knowledge, our method is able to directly search for architectures in the macro space by NSGA-II algorithm without tuning parameters in these textitmodules. Experiments show that our strategy can efficiently evaluate the performance of new architecture even without tuning weights in convolutional layers. With the help of knowledge we inherited, our search results can always achieve better performance on various datasets (CIFAR10, CIFAR100) over original architectures.