# A Novel Genetic Algorithm with Hierarchical Evaluation Strategy for Hyperparameter Optimisation of Graph Neural Networks

arXiv: https://arxiv.org/abs/2101.09300 · HTML: https://mlsanity.com/paper/2101.09300/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 50 references that are in the map.
- First posted on arXiv 22 Jan 2021.

## Builds on (most cited first, 19 of 19)

- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications](https://mlsanity.com/paper/1709.07604/) (2017)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Genetic CNN](https://mlsanity.com/paper/1703.01513/) (2017)
- [A Genetic Programming Approach to Designing Convolutional Neural Network Architectures](https://mlsanity.com/paper/1704.00764/) (2017)
- [Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search](https://mlsanity.com/paper/1807.06906/) (2018)
- [Hyperparameter Search in Machine Learning](https://mlsanity.com/paper/1502.02127/) (2015)
- [Deep Neural Architecture Search with Deep Graph Bayesian Optimization](https://mlsanity.com/paper/1905.06159/) (2019)
- [Graph Structural-topic Neural Network](https://mlsanity.com/paper/2006.14278/) (2020)
- [Neural Architecture Search in Graph Neural Networks](https://mlsanity.com/paper/2008.00077/) (2020)
- [Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm](https://mlsanity.com/paper/2006.12703/) (2020)
- [ImmuNeCS: Neural Committee Search by an Artificial Immune System](https://mlsanity.com/paper/1911.07729/) (2019)

## Built on by (most cited first, 7 of 7)

- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Automated Machine Learning on Graphs: A Survey](https://mlsanity.com/paper/2103.00742/) (2021)
- [Automated Graph Machine Learning: Approaches, Libraries, Benchmarks and Directions](https://mlsanity.com/paper/2201.01288/) (2022)
- [MetaGL: Evaluation-Free Selection of Graph Learning Models via Meta-Learning](https://mlsanity.com/paper/2206.09280/) (2022)
- [A Genetic Algorithm with Tree-structured Mutation for Hyperparameter Optimisation of Graph Neural Networks](https://mlsanity.com/paper/2102.11995/) (2021)
- [Which Hyperparameters to Optimise? An Investigation of Evolutionary Hyperparameter Optimisation in Graph Neural Network For Molecular Property Prediction](https://mlsanity.com/paper/2104.06046/) (2021)
- [SAIS: A Novel Bio-Inspired Artificial Immune System Based on Symbiotic Paradigm](https://mlsanity.com/paper/2402.07244/) (2024)

## Abstract (arXiv)

Graph representation of structured data can facilitate the extraction of stereoscopic features, and it has demonstrated excellent ability when working with deep learning systems, the so-called Graph Neural Networks (GNNs). Choosing a promising architecture for constructing GNNs can be transferred to a hyperparameter optimisation problem, a very challenging task due to the size of the underlying search space and high computational cost for evaluating candidate GNNs. To address this issue, this research presents a novel genetic algorithm with a hierarchical evaluation strategy (HESGA), which combines the full evaluation of GNNs with a fast evaluation approach. By using full evaluation, a GNN is represented by a set of hyperparameter values and trained on a specified dataset, and root mean square error (RMSE) will be used to measure the quality of the GNN represented by the set of hyperparameter values (for regression problems). While in the proposed fast evaluation process, the training will be interrupted at an early stage, the difference of RMSE values between the starting and interrupted epochs will be used as a fast score, which implies the potential of the GNN being considered. To coordinate both types of evaluations, the proposed hierarchical strategy uses the fast evaluation in a lower level for recommending candidates to a higher level, where the full evaluation will act as a final assessor to maintain a group of elite individuals. To validate the effectiveness of HESGA, we apply it to optimise two types of deep graph neural networks. The experimental results on three benchmark datasets demonstrate its advantages compared to Bayesian hyperparameter optimization.