# Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1807.07362 · HTML: https://mlsanity.com/paper/1807.07362/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jul 2018.

## Built on by (most cited first, 10 of 10)

- [Multi-Agent Automated Machine Learning](https://mlsanity.com/paper/2210.09084/) (2022)
- [Generalization of CNNs on Relational Reasoning with Bar Charts](https://mlsanity.com/paper/2503.00086/) (2025)
- [Using explainability to design physics-aware CNNs for solving subsurface inverse problems](https://mlsanity.com/paper/2211.08651/) (2022)
- [Exploring the Performance of Pruning Methods in Neural Networks: An Empirical Study of the Lottery Ticket Hypothesis](https://mlsanity.com/paper/2303.15479/) (2023)
- [Hyperellipsoid Density Sampling: Exploitative Sequences to Accelerate High-Dimensional Numerical Optimization](https://mlsanity.com/paper/2511.07836/) (2025)
- [Genealogical Population-Based Training for Hyperparameter Optimization](https://mlsanity.com/paper/2109.14925/) (2021)
- [Early Diagnosis of Retinal Blood Vessel Damage via Deep Learning-Powered Collective Intelligence Models](https://mlsanity.com/paper/2210.09449/) (2022)
- [The Planner Optimization Problem: Formulations and Frameworks](https://mlsanity.com/paper/2303.06768/) (2023)
- [Improving Classification of Retinal Fundus Image Using Flow Dynamics Optimized Deep Learning Methods](https://mlsanity.com/paper/2305.00294/) (2023)
- [Approach to Finding a Robust Deep Learning Model](https://mlsanity.com/paper/2505.17254/) (2025)

## Abstract (arXiv)

Most learning algorithms require the practitioner to manually set the values of many hyperparameters before the learning process can begin. However, with modern algorithms, the evaluation of a given hyperparameter setting can take a considerable amount of time and the search space is often very high-dimensional. We suggest using a lower-dimensional representation of the original data to quickly identify promising areas in the hyperparameter space. This information can then be used to initialize the optimization algorithm for the original, higher-dimensional data. We compare this approach with the standard procedure of optimizing the hyperparameters only on the original input. We perform experiments with various state-of-the-art hyperparameter optimization algorithms such as random search, the tree of parzen estimators (TPEs), sequential model-based algorithm configuration (SMAC), and a genetic algorithm (GA). Our experiments indicate that it is possible to speed up the optimization process by using lower-dimensional data representations at the beginning, while increasing the dimensionality of the input later in the optimization process. This is independent of the underlying optimization procedure, making the approach promising for many existing hyperparameter optimization algorithms.