# Investigating Hyperparameter Optimization and Transferability for ES-HyperNEAT: A TPE Approach

arXiv: https://arxiv.org/abs/2609.00449 · HTML: https://mlsanity.com/paper/2609.00449/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 21 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 3 of 3)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)

## Built on by (most cited first, 3 of 3)

- [Per-Node Activation Function Evolution in Indirectly Encoded Substrates: Solvability, Limits, and Emergent Diversity](https://mlsanity.com/paper/2610.00149/) (2026)
- [Breaking the Central Bias: Spatially Partitioned Experts for Coordinate-Based Neuroevolution](https://mlsanity.com/paper/2609.11518/) (2026)
- [Early-Stopping Thresholds for ES-HyperNEAT: A Data-Driven Approach from Fitness Dynamics](https://mlsanity.com/paper/2609.13533/) (2026)

## Abstract (arXiv)

Neuroevolution of Augmenting Topologies (NEAT) and its advanced version, Evolvable-Substrate HyperNEAT (ES-HyperNEAT), have shown great potential in developing neural networks. However, their effectiveness heavily depends on the selection of hyperparameters. This study investigates the optimization of ES-HyperNEAT hyperparameters using the Tree-structured Parzen Estimator (TPE) on the MNIST classification task, exploring a search space of over 3 billion potential combinations. TPE effectively navigates this vast space, significantly outperforming random search in terms of mean, median, and best accuracy. During the validation process, the best hyperparameter configuration found by TPE achieves an accuracy of 29.00% on MNIST, surpassing previous studies while using a smaller population size and fewer generations. The transferability of the optimized hyperparameters is explored in logic operations and Fashion-MNIST tasks, revealing successful transfer to the more complex Fashion-MNIST problem but limited to simpler logic operations. This study emphasizes a method to unlock the full potential of neuroevolutionary algorithms and provides insights into the hyperparameters' transferability across tasks of varying complexity.