# Improving One-shot NAS by Suppressing the Posterior Fading

arXiv: https://arxiv.org/abs/1910.02543 · HTML: https://mlsanity.com/paper/1910.02543/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2019.

## Built on by (most cited first, 28 of 28)

- [BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2103.12424/) (2021)
- [NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search](https://mlsanity.com/paper/2105.14444/) (2021)
- [GLiT: Neural Architecture Search for Global and Local Image Transformer](https://mlsanity.com/paper/2107.02960/) (2021)
- [Searching the Search Space of Vision Transformer](https://mlsanity.com/paper/2111.14725/) (2021)
- [ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search](https://mlsanity.com/paper/2105.10154/) (2021)
- [ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild](https://mlsanity.com/paper/2208.11547/) (2022)
- [Auto-HeG: Automated Graph Neural Network on Heterophilic Graphs](https://mlsanity.com/paper/2302.12357/) (2023)
- [BN-NAS: Neural Architecture Search with Batch Normalization](https://mlsanity.com/paper/2108.07375/) (2021)
- [Analyzing and Mitigating Interference in Neural Architecture Search](https://mlsanity.com/paper/2108.12821/) (2021)
- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)
- [Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift](https://mlsanity.com/paper/2108.09671/) (2021)
- [One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking](https://mlsanity.com/paper/2104.00597/) (2021)
- [Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing](https://mlsanity.com/paper/2210.04153/) (2022)
- [Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search](https://mlsanity.com/paper/2104.05309/) (2021)
- [ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks](https://mlsanity.com/paper/2205.08119/) (2022)
- [β-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search](https://mlsanity.com/paper/2301.06393/) (2023)
- [Stimulative Training++: Go Beyond The Performance Limits of Residual Networks](https://mlsanity.com/paper/2305.02507/) (2023)
- [BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule](https://mlsanity.com/paper/2111.13204/) (2021)
- [Vertical Layering of Quantized Neural Networks for Heterogeneous Inference](https://mlsanity.com/paper/2212.05326/) (2022)
- [Lightweight Diffusion Models with Distillation-Based Block Neural Architecture Search](https://mlsanity.com/paper/2311.04950/) (2023)
- [DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions](https://mlsanity.com/paper/2403.01326/) (2024)
- [An Analysis of Super-Net Heuristics in Weight-Sharing NAS](https://mlsanity.com/paper/2110.01154/) (2021)
- [Neural Architecture Searching for Facial Attributes-based Depression Recognition](https://mlsanity.com/paper/2201.09799/) (2022)
- [CARL: Causality-guided Architecture Representation Learning for an Interpretable Performance Predictor](https://mlsanity.com/paper/2506.04001/) (2025)
- [Auto Machine Learning for Medical Image Analysis by Unifying the Search on Data Augmentation and Neural Architecture](https://mlsanity.com/paper/2207.10351/) (2022)
- [SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference](https://mlsanity.com/paper/2303.08308/) (2023)
- [Tiered Pruning for Efficient Differentialble Inference-Aware Neural Architecture Search](https://mlsanity.com/paper/2209.11785/) (2022)
- [AdaNAS: Adaptively Post-processing with Self-supervised Neural Architecture Search for Ensemble Rainfall Forecasts](https://mlsanity.com/paper/2312.16046/) (2023)

## Abstract (arXiv)

There is a growing interest in automated neural architecture search (NAS). To improve the efficiency of NAS, previous approaches adopt weight sharing method to force all models share the same set of weights. However, it has been observed that a model performing better with shared weights does not necessarily perform better when trained alone. In this paper, we analyse existing weight sharing one-shot NAS approaches from a Bayesian point of view and identify the posterior fading problem, which compromises the effectiveness of shared weights. To alleviate this problem, we present a practical approach to guide the parameter posterior towards its true distribution. Moreover, a hard latency constraint is introduced during the search so that the desired latency can be achieved. The resulted method, namely Posterior Convergent NAS (PC-NAS), achieves state-of-the-art performance under standard GPU latency constraint on ImageNet. In our small search space, our model PC-NAS-S attains 76.8 % top-1 accuracy, 2.1% higher than MobileNetV2 (1.4x) with the same latency. When adopted to the large search space, PC-NAS-L achieves 78.1 % top-1 accuracy within 11ms. The discovered architecture also transfers well to other computer vision applications such as object detection and person re-identification.