mlsanityData to 14 Oct 2026

Surrogate Model Based Hyperparameter Tuning for Deep Learning with SPOT

arXiv 2105.14625 ↗First posted 30 May 2021cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

A surrogate model based hyperparameter tuning approach for deep learning is presented. This article demonstrates how the architecture-level parameters (hyperparameters) of deep learning models that were implemented in Keras/tensorflow can be optimized. The implementation of the tuning procedure is 100% accessible from R, the software environment for statistical computing. With a few lines of code, existing R packages (tfruns and SPOT) can be combined to perform hyperparameter tuning. An elementary hyperparameter tuning task (neural network and the MNIST data) is used to exemplify this approach

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 73 references that are in the map. First posted on arXiv 30 May 2021.

Builds on

25 of 73 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  4. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  5. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
  6. 2014OpenML: networked science in machine learningcited by 526
  7. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  8. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  9. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 254
  10. 2017The Marginal Value of Adaptive Gradient Methods in Machine Learningcited by 250
  11. 2018Measuring the Effects of Data Parallelism on Neural Network Trainingcited by 193
  12. 2018Auto-Keras: An Efficient Neural Architecture Search Systemcited by 171
13 more
  1. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  2. 2020Descending through a Crowded Valley - Benchmarking Deep Learning Optimizerscited by 119
  3. 2019Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Modelcited by 117
  4. 2019On Empirical Comparisons of Optimizers for Deep Learningcited by 103
  5. 2018Tunability: Importance of Hyperparameters of Machine Learning Algorithmscited by 99
  6. 2019A Survey on Neural Architecture Searchcited by 71
  7. 2020Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDLcited by 67
  8. 2020Benchmarking in Optimization: Best Practice and Open Issuescited by 38
  9. 2019DeepOBS: A Deep Learning Optimizer Benchmark Suitecited by 37
  10. 2018Benchmarking Automatic Machine Learning Frameworkscited by 15
  11. 2017Critical Hyper-Parameters: No Random, No Crycited by 8
  12. 2018A Very Brief and Critical Discussion on AutoMLcited by 3
  13. 2020Cognitive Capabilities for the CAAI in Cyber-Physical Production Systemscited by 1

Built on by

Not cited yet.