mlsanityData to 14 Oct 2026

Deep Active Learning with Structured Neural Depth Search

arXiv 2306.02808 ↗First posted 5 Jun 2023cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Previous work optimizes traditional active learning (AL) processes with incremental neural network architecture search (Active-iNAS) based on data complexity change, which improves the accuracy and learning efficiency. However, Active-iNAS trains several models and selects the model with the best generalization performance for querying the subsequent samples after each active learning cycle. The independent training processes lead to an insufferable computational budget, which is significantly inefficient and limits search flexibility and final performance. To address this issue, we propose a novel active strategy with the method called structured variational inference (SVI) or structured neural depth search (SNDS) whereby we could use the gradient descent method in neural network depth search during AL processes. At the same time, we theoretically demonstrate that the current VI-based methods based on the mean-field assumption could lead to poor performance. We apply our strategy using three querying techniques and three datasets and show that our strategy outperforms current methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

22 of 30 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  4. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  5. 2016Variational Inference: A Review for Statisticianscited by 1,472
  6. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  7. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  8. 2020A Survey of Deep Active Learningcited by 554
  9. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  10. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 311
  11. 2017Advances in Variational Inferencecited by 236
  12. 2017Cost-Effective Active Learning for Deep Image Classificationcited by 166
10 more
  1. 2017Deep Active Learning over the Long Tailcited by 84
  2. 2020Depth Uncertainty in Neural Networkscited by 57
  3. 2019Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Datacited by 33
  4. 2015Efficient and Parsimonious Agnostic Active Learningcited by 17
  5. 2018Deep Active Learning with a Neural Architecture Searchcited by 17
  6. 2019Understanding Neural Architecture Search Techniquescited by 12
  7. 2022Variational Inference for Infinitely Deep Neural Networkscited by 10
  8. 2019Bayesian Learning of Neural Network Architecturescited by 5
  9. 2020Variational Depth Search in ResNetscited by 2
  10. 2022Auto Machine Learning for Medical Image Analysis by Unifying the Search on Data Augmentation and Neural Architecturecited by 1

Built on by

Not cited yet.