# Convolution Neural Network Hyperparameter Optimization Using Simplified Swarm Optimization

arXiv: https://arxiv.org/abs/2103.03995 · HTML: https://mlsanity.com/paper/2103.03995/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [An overview of deep learning in medical imaging focusing on MRI](https://mlsanity.com/paper/1811.10052/) (2018)
- [Hyperparameter Optimization: A Spectral Approach](https://mlsanity.com/paper/1706.00764/) (2017)
- [Advancements in Image Classification using Convolutional Neural Network](https://mlsanity.com/paper/1905.03288/) (2019)
- [Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm](https://mlsanity.com/paper/2006.12703/) (2020)
- [Deep Neural Network Hyperparameter Optimization with Orthogonal Array Tuning](https://mlsanity.com/paper/1907.13359/) (2019)
- [A New Loss Function for CNN Classifier Based on Pre-defined Evenly-Distributed Class Centroids](https://mlsanity.com/paper/1904.06008/) (2019)
- [Systematic Ensemble Model Selection Approach for Educational Data Mining](https://mlsanity.com/paper/2005.06647/) (2020)
- [Big Holes in Big Data: A Monte Carlo Algorithm for Detecting Large Hyper-rectangles in High Dimensional Data](https://mlsanity.com/paper/1704.00683/) (2017)

## Built on by (most cited first, 1 of 1)

- [Deep learning approach for interruption attacks detection in LEO satellite networks](https://mlsanity.com/paper/2301.03998/) (2022)

## Abstract (arXiv)

Convolutional neural networks (CNNs) are widely used in image recognition. Numerous CNN models, such as LeNet, AlexNet, VGG, ResNet, and GoogLeNet, have been proposed by increasing the number of layers, to improve the performance of CNNs. However, performance deteriorates beyond a certain number of layers. Hence, hyperparameter optimisation is a more efficient way to improve CNNs. To validate this concept, a new algorithm based on simplified swarm optimisation is proposed to optimise the hyperparameters of the simplest CNN model, which is LeNet. The results of experiments conducted on the MNIST, Fashion MNIST, and Cifar10 datasets showed that the accuracy of the proposed algorithm is higher than the original LeNet model and PSO-LeNet and that it has a high potential to be extended to more complicated models, such as AlexNet.