# DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning

arXiv: https://arxiv.org/abs/1905.13543 · HTML: https://mlsanity.com/paper/1905.13543/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 13 of 13)

- [StateFlow: Enhancing LLM Task-Solving through State-Driven Workflows](https://mlsanity.com/paper/2403.11322/) (2024)
- [Active Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2303.08348/) (2023)
- [OMPQ: Orthogonal Mixed Precision Quantization](https://mlsanity.com/paper/2109.07865/) (2021)
- [Cantor: Inspiring Multimodal Chain-of-Thought of MLLM](https://mlsanity.com/paper/2404.16033/) (2024)
- [Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective](https://mlsanity.com/paper/2303.11906/) (2023)
- [HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shifts](https://mlsanity.com/paper/2305.18421/) (2023)
- [How Many Validation Labels Do You Need? Exploring the Design Space of Label-Efficient Model Ranking](https://mlsanity.com/paper/2312.01619/) (2023)
- [Exploiting Lightweight Hierarchical ViT and Dynamic Framework for Efficient Visual Tracking](https://mlsanity.com/paper/2506.20381/) (2025)
- [General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Level](https://mlsanity.com/paper/2311.13892/) (2023)
- [Colab NAS: Obtaining lightweight task-specific convolutional neural networks following Occam's razor](https://mlsanity.com/paper/2212.07700/) (2022)
- [A Comprehensive Study of Structural Pruning for Vision Models](https://mlsanity.com/paper/2406.12315/) (2024)
- [Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment](https://mlsanity.com/paper/2411.00838/) (2024)
- [HEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioning](https://mlsanity.com/paper/2412.10723/) (2024)

## Abstract (arXiv)

Neural Architecture Search (NAS) has demonstrated state-of-the-art performance on various computer vision tasks. Despite the superior performance achieved, the efficiency and generality of existing methods are highly valued due to their high computational complexity and low generality. In this paper, we propose an efficient and unified NAS framework termed DDPNAS via dynamic distribution pruning, facilitating a theoretical bound on accuracy and efficiency. In particular, we first sample architectures from a joint categorical distribution. Then the search space is dynamically pruned and its distribution is updated every few epochs. With the proposed efficient network generation method, we directly obtain the optimal neural architectures on given constraints, which is practical for on-device models across diverse search spaces and constraints. The architectures searched by our method achieve remarkable top-1 accuracies, 97.56 and 77.2 on CIFAR-10 and ImageNet (mobile settings), respectively, with the fastest search process, i.e., only 1.8 GPU hours on a Tesla V100. Codes for searching and network generation are available at: https://openi.pcl.ac.cn/PCL AutoML/XNAS.