mlsanityData to 7 Oct 2026

Less is More: Proxy Datasets in NAS approaches

arXiv 2203.06905 ↗First posted 14 Mar 2022cs.LG cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Neural Architecture Search (NAS) defines the design of Neural Networks as a search problem. Unfortunately, NAS is computationally intensive because of various possibilities depending on the number of elements in the design and the possible connections between them. In this work, we extensively analyze the role of the dataset size based on several sampling approaches for reducing the dataset size (unsupervised and supervised cases) as an agnostic approach to reduce search time. We compared these techniques with four common NAS approaches in NAS-Bench-201 in roughly 1,400 experiments on CIFAR-100. One of our surprising findings is that in most cases we can reduce the amount of training data to 25%, consequently reducing search time to 25%, while at the same time maintaining the same accuracy as if training on the full dataset. Additionally, some designs derived from subsets out-perform designs derived from the full dataset by up to 22 p.p. accuracy.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 14 Mar 2022.

Builds on

28 of 35 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2014Going Deeper with Convolutionscited by 4,231
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  7. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  8. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  10. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  11. 2020Dataset Condensation with Gradient Matchingcited by 510
  12. 2017Progressive Neural Architecture Searchcited by 483
16 more
  1. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  2. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  3. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 323
  4. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  5. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  6. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  7. 2018Dataset Distillationcited by 224
  8. 2016A guide to convolution arithmetic for deep learningcited by 182
  9. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
  10. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  11. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
  12. 2019NAS evaluation is frustratingly hardcited by 69
  13. 2017A Genetic Programming Approach to Designing Convolutional Neural Network Architecturescited by 59
  14. 2019Using Small Proxy Datasets to Accelerate Hyperparameter Searchcited by 13
  15. 2019Stabilizing DARTS with Amended Gradient Estimation on Architectural Parameterscited by 11
  16. 2020NASTransfer: Analyzing Architecture Transferability in Large Scale Neural Architecture Searchcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 3; 2025: 4.

9 papers.

  1. 2025A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advancescited by 28
  2. 2024Unlocking Dataset Distillation with Diffusion Modelscited by 21
  3. 2022A Survey on Computationally Efficient Neural Architecture Searchcited by 12
  4. 2024A Study in Dataset Pruning for Image Super-Resolutioncited by 6
  5. 2024Distill the Best, Ignore the Rest: Improving Dataset Distillation with Loss-Value-Based Pruningcited by 5
  6. 2023ASP: Automatic Selection of Proxy dataset for efficient AutoMLcited by 3
  7. 2025HyperCore: Coreset Selection under Noise via Hypersphere Modelscited by 0
  8. 2025SubZeroCore: A Submodular Approach with Zero Training for Coreset Selectioncited by 0
  9. 2025Evolutionary Neural Architecture Search with Dual Contrastive Learningcited by 0