# Sub-Architecture Ensemble Pruning in Neural Architecture Search

arXiv: https://arxiv.org/abs/1910.00370 · HTML: https://mlsanity.com/paper/1910.00370/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2019.

## Built on by (most cited first, 7 of 7)

- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [LENAS: Learning-based Neural Architecture Search and Ensemble for 3D Radiotherapy Dose Prediction](https://mlsanity.com/paper/2106.06733/) (2021)
- [Optimizing Dense Feed-Forward Neural Networks](https://mlsanity.com/paper/2312.10560/) (2023)
- [Latency Adjustable Transformer Encoder for Language Understanding](https://mlsanity.com/paper/2201.03327/) (2022)
- [SQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networks](https://mlsanity.com/paper/2601.22711/) (2026)
- [Exploring Complicated Search Spaces with Interleaving-Free Sampling](https://mlsanity.com/paper/2112.02488/) (2021)
- [Incentive Mechanism Design for Distributed Ensemble Learning](https://mlsanity.com/paper/2310.08792/) (2023)

## Abstract (arXiv)

Neural architecture search (NAS) is gaining more and more attention in recent years due to its flexibility and remarkable capability to reduce the burden of neural network design. To achieve better performance, however, the searching process usually costs massive computations that might not be affordable for researchers and practitioners. While recent attempts have employed ensemble learning methods to mitigate the enormous computational cost, however, they neglect a key property of ensemble methods, namely diversity, which leads to collecting more similar sub-architectures with potential redundancy in the final design. To tackle this problem, we propose a pruning method for NAS ensembles called "Sub-Architecture Ensemble Pruning in Neural Architecture Search (SAEP)." It targets to leverage diversity and to achieve sub-ensemble architectures at a smaller size with comparable performance to ensemble architectures that are not pruned. Three possible solutions are proposed to decide which sub-architectures to prune during the searching process. Experimental results exhibit the effectiveness of the proposed method by largely reducing the number of sub-architectures without degrading the performance.