# Multinomial Distribution Learning for Effective Neural Architecture Search

arXiv: https://arxiv.org/abs/1905.07529 · HTML: https://mlsanity.com/paper/1905.07529/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2019.

## Built on by (most cited first, 17 of 17)

- [iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients](https://mlsanity.com/paper/2106.10784/) (2021)
- [Meta Architecture for Point Cloud Analysis](https://mlsanity.com/paper/2211.14462/) (2022)
- [OMPQ: Orthogonal Mixed Precision Quantization](https://mlsanity.com/paper/2109.07865/) (2021)
- [Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective](https://mlsanity.com/paper/2303.11906/) (2023)
- [You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient](https://mlsanity.com/paper/2106.02435/) (2021)
- [Task Difficulty Aware Parameter Allocation & Regularization for Lifelong Learning](https://mlsanity.com/paper/2304.05288/) (2023)
- [Reconsidering CO2 emissions from Computer Vision](https://mlsanity.com/paper/2104.08702/) (2021)
- [L2NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learning](https://mlsanity.com/paper/2109.12425/) (2021)
- [BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule](https://mlsanity.com/paper/2111.13204/) (2021)
- [Differentiable Architecture Search with Random Features](https://mlsanity.com/paper/2208.08835/) (2022)
- [Reusable Architecture Growth for Continual Stereo Matching](https://mlsanity.com/paper/2404.00360/) (2024)
- [TND-NAS: Towards Non-differentiable Objectives in Progressive Differentiable NAS Framework](https://mlsanity.com/paper/2111.03892/) (2021)
- [TS-ENAS:Two-Stage Evolution for Cell-based Network Architecture Search](https://mlsanity.com/paper/2310.09525/) (2023)
- [One-Shot Neural Architecture Search with Network Similarity Directed Initialization for Pathological Image Classification](https://mlsanity.com/paper/2506.14176/) (2025)
- [The UniNAS framework: combining modules in arbitrarily complex configurations with argument trees](https://mlsanity.com/paper/2112.01796/) (2021)
- [DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs](https://mlsanity.com/paper/2306.15390/) (2023)
- [Efficient Global Neural Architecture Search](https://mlsanity.com/paper/2502.03553/) (2025)

## Abstract (arXiv)

Architectures obtained by Neural Architecture Search (NAS) have achieved highly competitive performance in various computer vision tasks. However, the prohibitive computation demand of forward-backward propagation in deep neural networks and searching algorithms makes it difficult to apply NAS in practice. In this paper, we propose a Multinomial Distribution Learning for extremely effective NAS,which considers the search space as a joint multinomial distribution, i.e., the operation between two nodes is sampled from this distribution, and the optimal network structure is obtained by the operations with the most likely probability in this distribution. Therefore, NAS can be transformed to a multinomial distribution learning problem, i.e., the distribution is optimized to have a high expectation of the performance. Besides, a hypothesis that the performance ranking is consistent in every training epoch is proposed and demonstrated to further accelerate the learning process. Experiments on CIFAR10 and ImageNet demonstrate the effectiveness of our method. On CIFAR-10, the structure searched by our method achieves 2.55% test error, while being 6.0x (only 4 GPU hours on GTX1080Ti) faster compared with state-of-the-art NAS algorithms. On ImageNet, our model achieves 75.2% top1 accuracy under MobileNet settings (MobileNet V1/V2), while being 1.2x faster with measured GPU latency. Test code with pre-trained models are available at https://github.com/tanglang96/MDENAS