mlsanityData to 14 Oct 2026

ASP: Automatic Selection of Proxy dataset for efficient AutoML

arXiv 2310.11478 ↗First posted 17 Oct 2023cs.LG cs.AI cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have gained great success due to the increasing amounts of data, and diverse effective neural network designs. However, it also brings a heavy computing burden as the amount of training data is proportional to the training time. In addition, a well-behaved model requires repeated trials of different structure designs and hyper-parameters, which may take a large amount of time even with state-of-the-art (SOTA) hyper-parameter optimization (HPO) algorithms and neural architecture search (NAS) algorithms. In this paper, we propose an Automatic Selection of Proxy dataset framework (ASP) aimed to dynamically find the informative proxy subsets of training data at each epoch, reducing the training data size as well as saving the AutoML processing time. We verify the effectiveness and generalization of ASP on CIFAR10, CIFAR100, ImageNet16-120, and ImageNet-1k, across various public model benchmarks. The experiment results show that ASP can obtain better results than other data selection methods at all selection ratios. ASP can also enable much more efficient AutoML processing with a speedup of 2x-20x while obtaining better architectures and better hyper-parameters compared to utilizing the entire dataset.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 42 of its 57 references that are in the map. First posted on arXiv 17 Oct 2023.

Builds on

42 of 57 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  5. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  6. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  7. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  8. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  9. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  10. 2019Gradient based sample selection for online continual learningcited by 571
  11. 2020Dataset Condensation with Gradient Matchingcited by 516
  12. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
30 more
  1. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  2. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  3. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  4. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 405
  5. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 339
  6. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 328
  7. 2021GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Trainingcited by 274
  8. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  9. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 260
  10. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 254
  11. 2020GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learningcited by 226
  12. 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 221
  13. 2018A System for Massively Parallel Hyperparameter Tuningcited by 202
  14. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  15. 2019Accelerating Deep Learning by Focusing on the Biggest Loserscited by 94
  16. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  17. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
  18. 2020EcoNAS: Finding Proxies for Economical Neural Architecture Searchcited by 61
  19. 2021Efficient Lottery Ticket Finding: Less Data is Morecited by 44
  20. 2019E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savingscited by 36
  21. 2022AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuningcited by 22
  22. 2019AutoAssist: A Framework to Accelerate Training of Deep Neural Networkscited by 14
  23. 2021Accelerating Neural Architecture Search via Proxy Datacited by 14
  24. 2019Using Small Proxy Datasets to Accelerate Hyperparameter Searchcited by 13
  25. 2022Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scalecited by 13
  26. 2014Easy Hyperparameter Search Using Optunitycited by 11
  27. 2022Less is More: Proxy Datasets in NAS approachescited by 9
  28. 2021How Important is Importance Sampling for Deep Budgeted Training?cited by 5
  29. 2014Automated Machine Learning on Big Data using Stochastic Algorithm Tuningcited by 4
  30. 2019Data Proxy Generation for Fast and Efficient Neural Architecture Searchcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advancescited by 28
  2. 2025HyperCore: Coreset Selection under Noise via Hypersphere Modelscited by 0
  3. 2025SubZeroCore: A Submodular Approach with Zero Training for Coreset Selectioncited by 0