# Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm

arXiv: https://arxiv.org/abs/2006.12703 · HTML: https://mlsanity.com/paper/2006.12703/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2020.

## Built on by (most cited first, 13 of 13)

- [A Novel Genetic Algorithm with Hierarchical Evaluation Strategy for Hyperparameter Optimisation of Graph Neural Networks](https://mlsanity.com/paper/2101.09300/) (2021)
- [Hyperparameter Tuning for Deep Reinforcement Learning Applications](https://mlsanity.com/paper/2201.11182/) (2022)
- [Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approach](https://mlsanity.com/paper/2207.00128/) (2022)
- [Hyperparameter Optimization for Randomized Algorithms: A Case Study on Random Features](https://mlsanity.com/paper/2407.00584/) (2024)
- [Genetic CFL: Optimization of Hyper-Parameters in Clustered Federated Learning](https://mlsanity.com/paper/2107.07233/) (2021)
- [A Genetic Algorithm with Tree-structured Mutation for Hyperparameter Optimisation of Graph Neural Networks](https://mlsanity.com/paper/2102.11995/) (2021)
- [FedAVO: Improving Communication Efficiency in Federated Learning with African Vultures Optimizer](https://mlsanity.com/paper/2305.01154/) (2023)
- [Rolling the dice for better deep learning performance: A study of randomness techniques in deep neural networks](https://mlsanity.com/paper/2404.03992/) (2024)
- [Convolution Neural Network Hyperparameter Optimization Using Simplified Swarm Optimization](https://mlsanity.com/paper/2103.03995/) (2021)
- [HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization](https://mlsanity.com/paper/2110.01698/) (2021)
- [Genetic Algorithm based hyper-parameters optimization for transfer Convolutional Neural Network](https://mlsanity.com/paper/2103.03875/) (2021)
- [Intelligent gradient amplification for deep neural networks](https://mlsanity.com/paper/2305.18445/) (2023)
- [Evolutionary Optimization of 1D-CNN for Non-contact Respiration Pattern Classification](https://mlsanity.com/paper/2312.13035/) (2023)

## Abstract (arXiv)

Convolutional Neural Networks (CNN) have gained great success in many artificial intelligence tasks. However, finding a good set of hyperparameters for a CNN remains a challenging task. It usually takes an expert with deep knowledge, and trials and errors. Genetic algorithms have been used in hyperparameter optimizations. However, traditional genetic algorithms with fixed-length chromosomes may not be a good fit for optimizing deep learning hyperparameters, because deep learning models have variable number of hyperparameters depending on the model depth. As the depth increases, the number of hyperparameters grows exponentially, and searching becomes exponentially harder. It is important to have an efficient algorithm that can find a good model in reasonable time. In this article, we propose to use a variable length genetic algorithm (GA) to systematically and automatically tune the hyperparameters of a CNN to improve its performance. Experimental results show that our algorithm can find good CNN hyperparameters efficiently. It is clear from our experiments that if more time is spent on optimizing the hyperparameters, better results could be achieved. Theoretically, if we had unlimited time and CPU power, we could find the optimized hyperparameters and achieve the best results in the future.