# A Novel Surrogate-assisted Evolutionary Algorithm Applied to Partition-based Ensemble Learning

arXiv: https://arxiv.org/abs/2104.08048 · HTML: https://mlsanity.com/paper/2104.08048/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection](https://mlsanity.com/paper/1904.07392/) (2019)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [Combinatorial Bayesian Optimization using the Graph Cartesian Product](https://mlsanity.com/paper/1902.00448/) (2019)
- [Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1706.03673/) (2017)
- [Bayesian Optimization of Combinatorial Structures](https://mlsanity.com/paper/1806.08838/) (2018)
- [D-LEMA: Deep Learning Ensembles from Multiple Annotations -- Application to Skin Lesion Segmentation](https://mlsanity.com/paper/2012.07206/) (2020)
- [Local Search is a Remarkably Strong Baseline for Neural Architecture Search](https://mlsanity.com/paper/2004.08996/) (2020)
- [Observer variation-aware medical image segmentation by combining deep learning and surrogate-assisted genetic algorithms](https://mlsanity.com/paper/2001.08552/) (2020)

## Built on by (most cited first, 2 of 2)

- [Data variation-aware medical image segmentation](https://mlsanity.com/paper/2202.12099/) (2022)
- [Heed the Noise in Performance Evaluations in Neural Architecture Search](https://mlsanity.com/paper/2202.02078/) (2022)

## Abstract (arXiv)

We propose a novel surrogate-assisted Evolutionary Algorithm for solving expensive combinatorial optimization problems. We integrate a surrogate model, which is used for fitness value estimation, into a state-of-the-art P3-like variant of the Gene-Pool Optimal Mixing Algorithm (GOMEA) and adapt the resulting algorithm for solving non-binary combinatorial problems. We test the proposed algorithm on an ensemble learning problem. Ensembling several models is a common Machine Learning technique to achieve better performance. We consider ensembles of several models trained on disjoint subsets of a dataset. Finding the best dataset partitioning is naturally a combinatorial non-binary optimization problem. Fitness function evaluations can be extremely expensive if complex models, such as Deep Neural Networks, are used as learners in an ensemble. Therefore, the number of fitness function evaluations is typically limited, necessitating expensive optimization techniques. In our experiments we use five classification datasets from the OpenML-CC18 benchmark and Support-vector Machines as learners in an ensemble. The proposed algorithm demonstrates better performance than alternative approaches, including Bayesian optimization algorithms. It manages to find better solutions using just several thousand fitness function evaluations for an ensemble learning problem with up to 500 variables.