# Assessing the Generalizability of a Performance Predictive Model

arXiv: https://arxiv.org/abs/2306.00040 · HTML: https://mlsanity.com/paper/2306.00040/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 5 of 5)

- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learning](https://mlsanity.com/paper/1711.08921/) (2017)
- [Per-run Algorithm Selection with Warm-starting using Trajectory-based Features](https://mlsanity.com/paper/2204.09483/) (2022)
- [Landscape-Aware Fixed-Budget Performance Regression and Algorithm Selection for Modular CMA-ES Variants](https://mlsanity.com/paper/2006.09855/) (2020)
- [SELECTOR: Selecting a Representative Benchmark Suite for Reproducible Statistical Comparison](https://mlsanity.com/paper/2204.11527/) (2022)

## Built on by (most cited first, 4 of 4)

- [Generalization Ability of Feature-based Performance Prediction Models: A Statistical Analysis across Benchmarks](https://mlsanity.com/paper/2405.12259/) (2024)
- [A Survey of Features Used for Representing Black-box Single-objective Continuous Optimization](https://mlsanity.com/paper/2406.06629/) (2024)
- [Evaluating Real-World Generalizability of Algorithm Selection Models](https://mlsanity.com/paper/2606.02016/) (2026)
- [Unsupervised Multi-kernel Learning for Automated Algorithm Selection](https://mlsanity.com/paper/2607.19031/) (2026)

## Abstract (arXiv)

A key component of automated algorithm selection and configuration, which in most cases are performed using supervised machine learning (ML) methods is a good-performing predictive model. The predictive model uses the feature representation of a set of problem instances as input data and predicts the algorithm performance achieved on them. Common machine learning models struggle to make predictions for instances with feature representations not covered by the training data, resulting in poor generalization to unseen problems. In this study, we propose a workflow to estimate the generalizability of a predictive model for algorithm performance, trained on one benchmark suite to another. The workflow has been tested by training predictive models across benchmark suites and the results show that generalizability patterns in the landscape feature space are reflected in the performance space.