mlsanityData to 14 Oct 2026

Speeding up NAS with Adaptive Subset Selection

arXiv 2211.01454 ↗First posted 2 Nov 2022cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

A majority of recent developments in neural architecture search (NAS) have been aimed at decreasing the computational cost of various techniques without affecting their final performance. Towards this goal, several low-fidelity and performance prediction methods have been considered, including those that train only on subsets of the training data. In this work, we present an adaptive subset selection approach to NAS and present it as complementary to state-of-the-art NAS approaches. We uncover a natural connection between one-shot NAS algorithms and adaptive subset selection and devise an algorithm that makes use of state-of-the-art techniques from both areas. We use these techniques to substantially reduce the runtime of DARTS-PT (a leading one-shot NAS algorithm), as well as BOHB and DEHB (leading multifidelity optimization algorithms), without sacrificing accuracy. Our results are consistent across multiple datasets, and towards full reproducibility, we release our code at https: //anonymous.4open.science/r/SubsetSelection NAS-B132.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 38 of its 80 references that are in the map. First posted on arXiv 2 Nov 2022.

Builds on

38 of 80 references.

References · 38

  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  5. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  6. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  7. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  8. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  9. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  10. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  11. 2021GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Trainingcited by 274
  12. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
26 more
  1. 2020Neural Architecture Search without Trainingcited by 244
  2. 2020GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learningcited by 226
  3. 2018A System for Massively Parallel Hyperparameter Tuningcited by 202
  4. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  5. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 195
  6. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 179
  7. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  8. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  9. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 104
  10. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  11. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  12. 2020EcoNAS: Finding Proxies for Economical Neural Architecture Searchcited by 61
  13. 2021Efficient Lottery Ticket Finding: Less Data is Morecited by 44
  14. 2020Geometry-Aware Gradient Algorithms for Neural Architecture Searchcited by 40
  15. 2022Compute-Efficient Deep Learning: Algorithmic Trends and Opportunitiescited by 38
  16. 2020FBNetV3: Joint Architecture-Recipe Search using Predictor Pretrainingcited by 35
  17. 2021Towards Green Automated Machine Learning: Status Quo and Future Directionscited by 25
  18. 2021Core-set Sampling for Efficient Neural Architecture Searchcited by 23
  19. 2022AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuningcited by 22
  20. 2021NAS-Bench-x11 and the Power of Learning Curvescited by 20
  21. 2021Accelerating Neural Architecture Search via Proxy Datacited by 14
  22. 2022SuperTickets: Drawing Task-Agnostic Lottery Tickets from Supernets via Jointly Architecture Searching and Parameter Pruningcited by 12
  23. 2021It's Time to Do Something: Mitigating the Negative Impacts of Computing Through a Change to the Peer Review Processcited by 11
  24. 2022Submodlib: A Submodular Optimization Librarycited by 8
  25. 2020Synthetic Petri Dish: A Novel Surrogate Model for Rapid Architecture Searchcited by 3
  26. 2019Prune and Replace NAScited by 2

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 0; 2025: 3.

6 papers.

  1. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  2. 2023A Comprehensive Survey of Dataset Distillationcited by 115
  3. 2023Efficient Architecture Search via Bi-level Data Pruningcited by 2
  4. 2025MicroNAS: An Automated Framework for Developing a Fall Detection Systemcited by 1
  5. 2025DDS-NAS: Dynamic Data Selection within Neural Architecture Search via On-line Hard Example Mining applied to Image Classificationcited by 1
  6. 2025Evolutionary Neural Architecture Search with Dual Contrastive Learningcited by 1