# Augmenting Novelty Search with a Surrogate Model to Engineer Meta-Diversity in Ensembles of Classifiers

arXiv: https://arxiv.org/abs/2201.12896 · HTML: https://mlsanity.com/paper/2201.12896/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 23 references that are in the map.
- First posted on arXiv 30 Jan 2022.

## Builds on (most cited first, 7 of 7)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Peephole: Predicting Network Performance Before Training](https://mlsanity.com/paper/1712.03351/) (2017)
- [Data-efficient Neuroevolution with Kernel-Based Surrogate Models](https://mlsanity.com/paper/1804.05364/) (2018)
- [Improving NeuroEvolution Efficiency by Surrogate Model-based Optimization with Phenotypic Distance Kernels](https://mlsanity.com/paper/1902.03419/) (2019)
- [Prediction of neural network performance by phenotypic modeling](https://mlsanity.com/paper/1907.07075/) (2019)
- [Surrogate Assisted Evolutionary Algorithm for Medium Scale Expensive Multi-Objective Optimisation Problems](https://mlsanity.com/paper/2002.03150/) (2020)

## Abstract (arXiv)

Using Neuroevolution combined with Novelty Search to promote behavioural diversity is capable of constructing high-performing ensembles for classification. However, using gradient descent to train evolved architectures during the search can be computationally prohibitive. Here we propose a method to overcome this limitation by using a surrogate model which estimates the behavioural distance between two neural network architectures required to calculate the sparseness term in Novelty Search. We demonstrate a speedup of 10 times over previous work and significantly improve on previous reported results on three benchmark datasets from Computer Vision -- CIFAR-10, CIFAR-100, and SVHN. This results from the expanded architecture search space facilitated by using a surrogate. Our method represents an improved paradigm for implementing horizontal scaling of learning algorithms by making an explicit search for diversity considerably more tractable for the same bounded resources.