mlsanityData to 7 Oct 2026

HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization

arXiv 2110.01698 ↗First posted 4 Oct 2021cs.LG cs.DCcited by 1

Not on the map. Why?

Abstract (arXiv)

We present a new software, HYPPO, that enables the automatic tuning of hyperparameters of various deep learning (DL) models. Unlike other hyperparameter optimization (HPO) methods, HYPPO uses adaptive surrogate models and directly accounts for uncertainty in model predictions to find accurate and reliable models that make robust predictions. Using asynchronous nested parallelism, we are able to significantly alleviate the computational burden of training complex architectures and quantifying the uncertainty. HYPPO is implemented in Python and can be used with both TensorFlow and PyTorch libraries. We demonstrate various software features on time-series prediction and image classification problems as well as a scientific application in computed tomography image reconstruction. Finally, we show that (1) we can reduce by an order of magnitude the number of evaluations necessary to find the most optimal region in the hyperparameter space and (2) we can reduce by two orders of magnitude the throughput for such HPO process to complete.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 14 of its 49 references that are in the map. First posted on arXiv 4 Oct 2021.

Builds on

14 of 49 references.

References · 14

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  4. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  5. 2017Population Based Training of Neural Networkscited by 351
  6. 2019Survey of Dropout Methods for Deep Neural Networkscited by 44
  7. 2018Radon Inversion via Deep Learningcited by 35
  8. 2020Using Machine Learning to Augment Coarse-Grid Computational Fluid Dynamics Simulationscited by 24
  9. 2020Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithmcited by 13
  10. 2018Illumination Pattern Design with Deep Learning for Single-Shot Fourier Ptychographic Microscopycited by 9
  11. 2019Surrogate Optimization of Deep Neural Networks for Groundwater Predictionscited by 6
  12. 2019A hierarchical approach to deep learning and its application to tomographic reconstructioncited by 5
2 more
  1. 2019Deep Learning Based Computed Tomography Whys and Whereforescited by 2
  2. 2018Optimal Physical Preprocessing for Example-Based Super-Resolutioncited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Hyperparameter Optimization of Generative Adversarial Network Models for High-Energy Physics Simulationscited by 0