# Deep Neural Network Hyperparameter Optimization with Orthogonal Array Tuning

arXiv: https://arxiv.org/abs/1907.13359 · HTML: https://mlsanity.com/paper/1907.13359/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2019.

## Built on by (most cited first, 11 of 11)

- [Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization](https://mlsanity.com/paper/2106.06916/) (2021)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programs](https://mlsanity.com/paper/2205.01938/) (2022)
- [Invariance encoding in sliced-Wasserstein space for image classification with limited training data](https://mlsanity.com/paper/2201.02980/) (2022)
- [Neural Networks Optimized by Genetic Algorithms in Cosmology](https://mlsanity.com/paper/2209.02685/) (2022)
- [ECG-Based Heart Arrhythmia Diagnosis Through Attentional Convolutional Neural Networks](https://mlsanity.com/paper/2108.10226/) (2021)
- [Convolution Neural Network Hyperparameter Optimization Using Simplified Swarm Optimization](https://mlsanity.com/paper/2103.03995/) (2021)
- [Optimized Three Deep Learning Models Based-PSO Hyperparameters for Beijing PM2.5 Prediction](https://mlsanity.com/paper/2306.07296/) (2023)
- [Ensemble CNN and Uncertainty Modeling to Improve Automatic Identification/Segmentation of Multiple Sclerosis Lesions in Magnetic Resonance Imaging](https://mlsanity.com/paper/2108.11791/) (2021)
- [Optimal Hyperparameters and Structure Setting of Multi-Objective Robust CNN Systems via Generalized Taguchi Method and Objective Vector Norm](https://mlsanity.com/paper/2202.04567/) (2022)
- [Dense Optimizer : An Information Entropy-Guided Structural Search Method for Dense-like Neural Network Design](https://mlsanity.com/paper/2410.07499/) (2024)

## Abstract (arXiv)

Deep learning algorithms have achieved excellent performance lately in a wide range of fields (e.g., computer version). However, a severe challenge faced by deep learning is the high dependency on hyper-parameters. The algorithm results may fluctuate dramatically under the different configuration of hyper-parameters. Addressing the above issue, this paper presents an efficient Orthogonal Array Tuning Method (OATM) for deep learning hyper-parameter tuning. We describe the OATM approach in five detailed steps and elaborate on it using two widely used deep neural network structures (Recurrent Neural Networks and Convolutional Neural Networks). The proposed method is compared to the state-of-the-art hyper-parameter tuning methods including manually (e.g., grid search and random search) and automatically (e.g., Bayesian Optimization) ones. The experiment results state that OATM can significantly save the tuning time compared to the state-of-the-art methods while preserving the satisfying performance. The codes are open in GitHub (https://github.com/xiangzhang1015/OATM)