# NeuroLGP-SM: Scalable Surrogate-Assisted Neuroevolution for Deep Neural Networks

arXiv: https://arxiv.org/abs/2404.08786 · HTML: https://mlsanity.com/paper/2404.08786/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 12 Apr 2024.

## Builds on (most cited first, 17 of 17)

- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better](https://mlsanity.com/paper/2106.08962/) (2021)
- [Freeze-Thaw Bayesian Optimization](https://mlsanity.com/paper/1406.3896/) (2014)
- [Evolving Deep Convolutional Neural Networks for Image Classification](https://mlsanity.com/paper/1710.10741/) (2017)
- [Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges](https://mlsanity.com/paper/2006.05415/) (2020)
- [Data-efficient Neuroevolution with Kernel-Based Surrogate Models](https://mlsanity.com/paper/1804.05364/) (2018)
- [Improving NeuroEvolution Efficiency by Surrogate Model-based Optimization with Phenotypic Distance Kernels](https://mlsanity.com/paper/1902.03419/) (2019)
- [Prediction of neural network performance by phenotypic modeling](https://mlsanity.com/paper/1907.07075/) (2019)
- [Neuroevolutionary Multi-objective approaches to Trajectory Prediction in Autonomous Vehicles](https://mlsanity.com/paper/2205.02105/) (2022)
- [Diving Deep onto Discriminative Ensemble of Histological Hashing & Class-Specific Manifold Learning for Multi-class Breast Carcinoma Taxonomy](https://mlsanity.com/paper/1806.06876/) (2018)
- [Semantics in Multi-objective Genetic Programming](https://mlsanity.com/paper/2105.02944/) (2021)
- [Semantic-based Distance Approaches in Multi-objective Genetic Programming](https://mlsanity.com/paper/2009.12401/) (2020)
- [Semantic Neighborhood Ordering in Multi-objective Genetic Programming based on Decomposition](https://mlsanity.com/paper/2103.00480/) (2021)
- [Initial Steps Towards Tackling High-dimensional Surrogate Modeling for Neuroevolution Using Kriging Partial Least Squares](https://mlsanity.com/paper/2305.03612/) (2023)
- [Evolutionary Multi-objective Optimisation in Neurotrajectory Prediction](https://mlsanity.com/paper/2308.02710/) (2023)
- [NeuroLGP-SM: A Surrogate-assisted Neuroevolution Approach using Linear Genetic Programming](https://mlsanity.com/paper/2403.19459/) (2024)
- [Neural Architecture Search Using Genetic Algorithm for Facial Expression Recognition](https://mlsanity.com/paper/2304.12194/) (2023)

## Built on by (most cited first, 1 of 1)

- [Zero-Shot LLMs in Human-in-the-Loop RL: Replacing Human Feedback for Reward Shaping](https://mlsanity.com/paper/2503.22723/) (2025)

## Abstract (arXiv)

Evolutionary Algorithms (EAs) play a crucial role in the architectural configuration and training of Artificial Deep Neural Networks (DNNs), a process known as neuroevolution. However, neuroevolution is hindered by its inherent computational expense, requiring multiple generations, a large population, and numerous epochs. The most computationally intensive aspect lies in evaluating the fitness function of a single candidate solution. To address this challenge, we employ Surrogate-assisted EAs (SAEAs). While a few SAEAs approaches have been proposed in neuroevolution, none have been applied to truly large DNNs due to issues like intractable information usage. In this work, drawing inspiration from Genetic Programming semantics, we use phenotypic distance vectors, outputted from DNNs, alongside Kriging Partial Least Squares (KPLS), an approach that is effective in handling these large vectors, making them suitable for search. Our proposed approach, named Neuro-Linear Genetic Programming surrogate model (NeuroLGP-SM), efficiently and accurately estimates DNN fitness without the need for complete evaluations. NeuroLGP-SM demonstrates competitive or superior results compared to 12 other methods, including NeuroLGP without SM, convolutional neural networks, support vector machines, and autoencoders. Additionally, it is worth noting that NeuroLGP-SM is 25% more energy-efficient than its NeuroLGP counterpart. This efficiency advantage adds to the overall appeal of our proposed NeuroLGP-SM in optimising the configuration of large DNNs.