mlsanityData to 7 Oct 2026

Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search

arXiv 1807.06906 ↗First posted 18 Jul 2018cs.LG cs.AI cs.CVcited by 38

Not on the map. Why?

Abstract (arXiv)

While existing work on neural architecture search (NAS) tunes hyperparameters in a separate post-processing step, we demonstrate that architectural choices and other hyperparameter settings interact in a way that can render this separation suboptimal. Likewise, we demonstrate that the common practice of using very few epochs during the main NAS and much larger numbers of epochs during a post-processing step is inefficient due to little correlation in the relative rankings for these two training regimes. To combat both of these problems, we propose to use a recent combination of Bayesian optimization and Hyperband for efficient joint neural architecture and hyperparameter search.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 15; 2022: 10; 2023: 8; 2024: 2; 2025: 3.

38 papers.

Sort:

  1. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 143
  2. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  3. 2021Scaling Laws for Deep Learningcited by 25
  4. 2021PyGlove: Symbolic Programming for Automated Machine Learningcited by 19
  5. 2021Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimizationcited by 19
  6. 2023Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoMLcited by 16
  7. 2021AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Flycited by 14
  8. 2022Efficient Automated Deep Learning for Time Series Forecastingcited by 14
  9. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  10. 2022A Survey on Computationally Efficient Neural Architecture Searchcited by 12
  11. 2022Neural Architecture Search for Dense Prediction Tasks in Computer Visioncited by 11
  12. 2022PRE-NAS: Predictor-assisted Evolutionary Neural Architecture Searchcited by 8
26 more
  1. 2025Scaling with Collapse: Efficient and Predictable Training of LLM Familiescited by 8
  2. 2021A Novel Genetic Algorithm with Hierarchical Evaluation Strategy for Hyperparameter Optimisation of Graph Neural Networkscited by 7
  3. 2021Adversarial Branch Architecture Search for Unsupervised Domain Adaptationcited by 6
  4. 2021DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecturecited by 6
  5. 2021DAAS: Differentiable Architecture and Augmentation Policy Searchcited by 4
  6. 2022Towards Less Constrained Macro-Neural Architecture Searchcited by 4
  7. 2022Exploring the Intersection between Neural Architecture Search and Continual Learningcited by 4
  8. 2023AutoML for Large Capacity Modeling of Meta's Ranking Systemscited by 4
  9. 2024Principled Architecture-aware Scaling of Hyperparameterscited by 4
  10. 2021Accelerating Multi-Objective Neural Architecture Search by Random-Weight Evaluationcited by 3
  11. 2021AUTOKD: Automatic Knowledge Distillation Into A Student Architecture Familycited by 3
  12. 2022Multi-Agent Automated Machine Learningcited by 3
  13. 2023DP-HyPO: An Adaptive Private Hyperparameter Optimization Frameworkcited by 3
  14. 2023Efficient and Joint Hyperparameter and Architecture Search for Collaborative Filteringcited by 3
  15. 2025Auto-nnU-Net: Towards Automated Medical Image Segmentationcited by 3
  16. 2021AutoDrop: Training Deep Learning Models with Automatic Learning Rate Dropcited by 2
  17. 2022Improving Accuracy of Interpretability Measures in Hyperparameter Optimization via Bayesian Algorithm Executioncited by 2
  18. 2022Automatic Neural Network Hyperparameter Optimization for Extrapolation: Lessons Learned from Visible and Near-Infrared Spectroscopy of Mango Fruitcited by 2
  19. 2023Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Derainingcited by 2
  20. 2023HyperTuner: A Cross-Layer Multi-Objective Hyperparameter Auto-Tuning Framework for Data Analytic Servicescited by 1
  21. 2023DREAM: Debugging and Repairing AutoML Pipelinescited by 1
  22. 2024GRASP-GCN: Graph-Shape Prioritization for Neural Architecture Search under Distribution Shiftscited by 1
  23. 2021Use of static surrogates in hyperparameter optimizationcited by 0
  24. 2021GPNAS: A Neural Network Architecture Search Framework Based on Graphical Predictorcited by 0
  25. 2022UnrealNAS: Can We Search Neural Architectures with Unreal Data?cited by 0
  26. 2025You Only Train Oncecited by 0