DDS-NAS: Dynamic Data Selection within Neural Architecture Search via On-line Hard Example Mining applied to Image Classification
Not on the map. Why?
Abstract (arXiv)
In order to address the scalability challenge within Neural Architecture Search (NAS), we speed up NAS training via dynamic hard example mining within a curriculum learning framework. By utilizing an autoencoder that enforces an image similarity embedding in latent space, we construct an efficient kd-tree structure to order images by furthest neighbour dissimilarity in a low-dimensional embedding. From a given query image from our subsample dataset, we can identify the most dissimilar image within the global dataset in logarithmic time. Via curriculum learning, we then dynamically re-formulate an unbiased subsample dataset for NAS optimisation, upon which the current NAS solution architecture performs poorly. We show that our DDS-NAS framework speeds up gradient-based NAS strategies by up to 27x without loss in performance. By maximising the contribution of each image sample during training, we reduce the duration of a NAS training cycle and the number of iterations required for convergence.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 17 Jun 2025.
Builds on
32 of 50 references.
References · 32
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
- 2017Deep Image Priorcited by 1,098
- 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
- 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 579
- 2022Dataset Distillation by Matching Training Trajectoriescited by 442
20 more
- 2018GANomaly: Semi-Supervised Anomaly Detection via Adversarial Trainingcited by 409
- 2018SNAS: Stochastic Neural Architecture Searchcited by 289
- 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
- 2016Designing Neural Network Architectures using Reinforcement Learningcited by 279
- 2017Automated Curriculum Learning for Neural Networkscited by 257
- 2020GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learningcited by 226
- 2017Teacher-Student Curriculum Learningcited by 210
- 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 114
- 2019Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detectioncited by 100
- 2021Rethinking Architecture Selection in Differentiable NAScited by 92
- 2019Network Pruning via Transformable Architecture Searchcited by 80
- 2019Progressive DARTS: Bridging the Optimization Gap for NAS in the Wildcited by 40
- 2020Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Searchcited by 33
- 2021Core-set Sampling for Efficient Neural Architecture Searchcited by 23
- 2022Shapley-NAS: Discovering Operation Contribution for Neural Architecture Searchcited by 23
- 2024A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trendscited by 16
- 2021Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shiftcited by 14
- 2022Speeding up NAS with Adaptive Subset Selectioncited by 6
- 2022CLOSE: Curriculum Learning On the Sharing Extent Towards Better One-shot NAScited by 5
- 2024Scalable Reinforcement Learning-based Neural Architecture Searchcited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.