mlsanityData to 7 Oct 2026

Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search

arXiv 2104.05309 ↗First posted 12 Apr 2021cs.LG cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Weight sharing has become a de facto standard in neural architecture search because it enables the search to be done on commodity hardware. However, recent works have empirically shown a ranking disorder between the performance of stand-alone architectures and that of the corresponding shared-weight networks. This violates the main assumption of weight-sharing NAS algorithms, thus limiting their effectiveness. We tackle this issue by proposing a regularization term that aims to maximize the correlation between the performance rankings of the shared-weight network and that of the standalone architectures using a small set of landmark architectures. We incorporate our regularization term into three different NAS algorithms and show that it consistently improves performance across algorithms, search-spaces, and tasks.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 49 of its 57 references that are in the map. First posted on arXiv 12 Apr 2021.

Builds on

49 of 57 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  5. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  7. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  8. 2020Designing Network Design Spacescited by 954
  9. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  10. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  11. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  12. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
37 more
  1. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  2. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  3. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  4. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  5. 2017Large-Scale Evolution of Image Classifierscited by 348
  6. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  7. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  8. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  9. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  10. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  11. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
  12. 2020BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Modelscited by 187
  13. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
  14. 2018Neural Architecture Optimizationcited by 164
  15. 2018High Quality Monocular Depth Estimation via Transfer Learningcited by 133
  16. 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 129
  17. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 129
  18. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  19. 2020NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Sizecited by 98
  20. 2019NAS evaluation is frustratingly hardcited by 69
  21. 2020GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernetcited by 69
  22. 2019On Network Design Spaces for Visual Recognitioncited by 68
  23. 2020Can weight sharing outperform random architecture search? An investigation with TuNAScited by 68
  24. 2020NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Searchcited by 64
  25. 2018ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networkscited by 53
  26. 2020Few-shot Neural Architecture Searchcited by 50
  27. 2019AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architecturescited by 46
  28. 2020Semi-Supervised Neural Architecture Searchcited by 45
  29. 2020Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Searchcited by 38
  30. 2019XNAS: Neural Architecture Search with Expert Advicecited by 35
  31. 2019AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Searchcited by 33
  32. 2019Improving One-shot NAS by Suppressing the Posterior Fadingcited by 28
  33. 2020A Semi-Supervised Assessor of Neural Architecturescited by 28
  34. 2019Recurrent U-Net for Resource-Constrained Segmentationcited by 27
  35. 2020How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAScited by 17
  36. 2019Overcoming Multi-Model Forgettingcited by 14
  37. 2018Statistically Motivated Second Order Poolingcited by 5

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 5; 2023: 1.

8 papers.

  1. 2021NAS-Bench-x11 and the Power of Learning Curvescited by 20
  2. 2023RD-NAS: Enhancing One-shot Supernet Ranking Ability via Ranking Distillation from Zero-cost Proxiescited by 7
  3. 2022Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Searchcited by 5
  4. 2021Guided Evolution for Neural Architecture Searchcited by 3
  5. 2022Guided Evolutionary Neural Architecture Search With Efficient Performance Estimationcited by 2
  6. 2022AutoTS: Automatic Time Series Forecasting Model Design Based on Two-Stage Pruningcited by 1
  7. 2022Improve Ranking Correlation of Super-net through Training Scheme from One-shot NAS to Few-shot NAScited by 1
  8. 2022WPNAS: Neural Architecture Search by jointly using Weight Sharing and Predictorcited by 0