# NeuroLGP-SM: A Surrogate-assisted Neuroevolution Approach using Linear Genetic Programming

arXiv: https://arxiv.org/abs/2403.19459 · HTML: https://mlsanity.com/paper/2403.19459/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 28 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks](https://mlsanity.com/paper/2303.03667/) (2023)
- [Evolving Deep Neural Networks](https://mlsanity.com/paper/1703.00548/) (2017)
- [Freeze-Thaw Bayesian Optimization](https://mlsanity.com/paper/1406.3896/) (2014)
- [Evolving Deep Convolutional Neural Networks for Image Classification](https://mlsanity.com/paper/1710.10741/) (2017)
- [Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges](https://mlsanity.com/paper/2006.05415/) (2020)
- [Data-efficient Neuroevolution with Kernel-Based Surrogate Models](https://mlsanity.com/paper/1804.05364/) (2018)
- [Improving NeuroEvolution Efficiency by Surrogate Model-based Optimization with Phenotypic Distance Kernels](https://mlsanity.com/paper/1902.03419/) (2019)
- [Prediction of neural network performance by phenotypic modeling](https://mlsanity.com/paper/1907.07075/) (2019)
- [Neuroevolutionary Multi-objective approaches to Trajectory Prediction in Autonomous Vehicles](https://mlsanity.com/paper/2205.02105/) (2022)
- [Semantics in Multi-objective Genetic Programming](https://mlsanity.com/paper/2105.02944/) (2021)
- [Semantic-based Distance Approaches in Multi-objective Genetic Programming](https://mlsanity.com/paper/2009.12401/) (2020)
- [Semantic Neighborhood Ordering in Multi-objective Genetic Programming based on Decomposition](https://mlsanity.com/paper/2103.00480/) (2021)
- [Initial Steps Towards Tackling High-dimensional Surrogate Modeling for Neuroevolution Using Kriging Partial Least Squares](https://mlsanity.com/paper/2305.03612/) (2023)
- [Evolutionary Multi-objective Optimisation in Neurotrajectory Prediction](https://mlsanity.com/paper/2308.02710/) (2023)

## Built on by (most cited first, 2 of 2)

- [NeuroLGP-SM: Scalable Surrogate-Assisted Neuroevolution for Deep Neural Networks](https://mlsanity.com/paper/2404.08786/) (2024)
- [NeuroPareto: Calibrated Acquisition for Costly Many-Goal Search in Vast Parameter Spaces](https://mlsanity.com/paper/2602.03901/) (2026)

## Abstract (arXiv)

Evolutionary algorithms are increasingly recognised as a viable computational approach for the automated optimisation of deep neural networks (DNNs) within artificial intelligence. This method extends to the training of DNNs, an approach known as neuroevolution. However, neuroevolution is an inherently resource-intensive process, with certain studies reporting the consumption of thousands of GPU days for refining and training a single DNN network. To address the computational challenges associated with neuroevolution while still attaining good DNN accuracy, surrogate models emerge as a pragmatic solution. Despite their potential, the integration of surrogate models into neuroevolution is still in its early stages, hindered by factors such as the effective use of high-dimensional data and the representation employed in neuroevolution. In this context, we address these challenges by employing a suitable representation based on Linear Genetic Programming, denoted as NeuroLGP, and leveraging Kriging Partial Least Squares. The amalgamation of these two techniques culminates in our proposed methodology known as the NeuroLGP-Surrogate Model (NeuroLGP-SM). For comparison purposes, we also code and use a baseline approach incorporating a repair mechanism, a common practice in neuroevolution. Notably, the baseline approach surpasses the renowned VGG-16 model in accuracy. Given the computational intensity inherent in DNN operations, a singular run is typically the norm. To evaluate the efficacy of our proposed approach, we conducted 96 independent runs. Significantly, our methodologies consistently outperform the baseline, with the SM model demonstrating superior accuracy or comparable results to the NeuroLGP approach. Noteworthy is the additional advantage that the SM approach exhibits a 25% reduction in computational requirements, further emphasising its efficiency for neuroevolution.