# Generalization Ability of Feature-based Performance Prediction Models: A Statistical Analysis across Benchmarks

arXiv: https://arxiv.org/abs/2405.12259 · HTML: https://mlsanity.com/paper/2405.12259/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 40 references that are in the map.
- First posted on arXiv 20 May 2024.

## Builds on (most cited first, 11 of 11)

- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learning](https://mlsanity.com/paper/1711.08921/) (2017)
- [Comprehensive Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems Using the R-Package flacco](https://mlsanity.com/paper/1708.05258/) (2017)
- [IOHexperimenter: Benchmarking Platform for Iterative Optimization Heuristics](https://mlsanity.com/paper/2111.04077/) (2021)
- [Per-run Algorithm Selection with Warm-starting using Trajectory-based Features](https://mlsanity.com/paper/2204.09483/) (2022)
- [Landscape-Aware Fixed-Budget Performance Regression and Algorithm Selection for Modular CMA-ES Variants](https://mlsanity.com/paper/2006.09855/) (2020)
- [SELECTOR: Selecting a Representative Benchmark Suite for Reproducible Statistical Comparison](https://mlsanity.com/paper/2204.11527/) (2022)
- [BBOB Instance Analysis: Landscape Properties and Algorithm Performance across Problem Instances](https://mlsanity.com/paper/2211.16318/) (2022)
- [An Improved Distributed Algorithm for Maximal Independent Set](https://mlsanity.com/paper/1506.05093/) (2015)
- [Linear Matrix Factorization Embeddings for Single-objective Optimization Landscapes](https://mlsanity.com/paper/2009.14506/) (2020)
- [Assessing the Generalizability of a Performance Predictive Model](https://mlsanity.com/paper/2306.00040/) (2023)

## Built on by (most cited first, 4 of 4)

- [BLADE: Benchmark suite for LLM-driven Automated Design and Evolution of iterative optimisation heuristics](https://mlsanity.com/paper/2504.20183/) (2025)
- [Revisiting Generalization Power of a DNN in Terms of Symbolic Interactions](https://mlsanity.com/paper/2502.10162/) (2025)
- [Geometric Probing for Algorithm Selection in Continuous Black-Box Optimization](https://mlsanity.com/paper/2604.09095/) (2026)
- [Unsupervised Multi-kernel Learning for Automated Algorithm Selection](https://mlsanity.com/paper/2607.19031/) (2026)

## Abstract (arXiv)

This study examines the generalization ability of algorithm performance prediction models across various benchmark suites. Comparing the statistical similarity between the problem collections with the accuracy of performance prediction models that are based on exploratory landscape analysis features, we observe that there is a positive correlation between these two measures. Specifically, when the high-dimensional feature value distributions between training and testing suites lack statistical significance, the model tends to generalize well, in the sense that the testing errors are in the same range as the training errors. Two experiments validate these findings: one involving the standard benchmark suites, the BBOB and CEC collections, and another using five collections of affine combinations of BBOB problem instances.