# Improving Ranking Correlation of Supernet with Candidates Enhancement and Progressive Training

arXiv: https://arxiv.org/abs/2108.05866 · HTML: https://mlsanity.com/paper/2108.05866/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 7 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 6 of 6)

- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://mlsanity.com/paper/1904.00420/) (2019)
- [FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search](https://mlsanity.com/paper/1907.01845/) (2019)
- [MnasFPN: Learning Latency-aware Pyramid Architecture for Object Detection on Mobile Devices](https://mlsanity.com/paper/1912.01106/) (2019)
- [SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search](https://mlsanity.com/paper/1908.06022/) (2019)

## Built on by (most cited first, 2 of 2)

- [Prior-Guided One-shot Neural Architecture Search](https://mlsanity.com/paper/2206.13329/) (2022)
- [Improve Ranking Correlation of Super-net through Training Scheme from One-shot NAS to Few-shot NAS](https://mlsanity.com/paper/2206.05896/) (2022)

## Abstract (arXiv)

One-shot neural architecture search (NAS) applies weight-sharing supernet to reduce the unaffordable computation overhead of automated architecture designing. However, the weight-sharing technique worsens the ranking consistency of performance due to the interferences between different candidate networks. To address this issue, we propose a candidates enhancement method and progressive training pipeline to improve the ranking correlation of supernet. Specifically, we carefully redesign the sub-networks in the supernet and map the original supernet to a new one of high capacity. In addition, we gradually add narrow branches of supernet to reduce the degree of weight sharing which effectively alleviates the mutual interference between sub-networks. Finally, our method ranks the 1st place in the Supernet Track of CVPR2021 1st Lightweight NAS Challenge.