# BBOB Instance Analysis: Landscape Properties and Algorithm Performance across Problem Instances

arXiv: https://arxiv.org/abs/2211.16318 · HTML: https://mlsanity.com/paper/2211.16318/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2022.

## Built on by (most cited first, 10 of 10)

- [Explainable Benchmarking for Iterative Optimization Heuristics](https://mlsanity.com/paper/2401.17842/) (2024)
- [LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithms](https://mlsanity.com/paper/2505.21034/) (2025)
- [BLADE: Benchmark suite for LLM-driven Automated Design and Evolution of iterative optimisation heuristics](https://mlsanity.com/paper/2504.20183/) (2025)
- [MA-BBOB: Many-Affine Combinations of BBOB Functions for Evaluating AutoML Approaches in Noiseless Numerical Black-Box Optimization Contexts](https://mlsanity.com/paper/2306.10627/) (2023)
- [Generalization Ability of Feature-based Performance Prediction Models: A Statistical Analysis across Benchmarks](https://mlsanity.com/paper/2405.12259/) (2024)
- [The Pitfalls of Benchmarking in Algorithm Selection: What We Are Getting Wrong](https://mlsanity.com/paper/2505.07750/) (2025)
- [carps: A Framework for Comparing N Hyperparameter Optimizers on M Benchmarks](https://mlsanity.com/paper/2506.06143/) (2025)
- [Computational and Exploratory Landscape Analysis of the GKLS Generator](https://mlsanity.com/paper/2304.08913/) (2023)
- [Landscape-Aware Automated Algorithm Configuration using Multi-output Mixed Regression and Classification](https://mlsanity.com/paper/2409.01446/) (2024)
- [Beyond Numerical Features: CNN-Driven Algorithm Selection via Contour Plots for Continuous Black-Box Optimization](https://mlsanity.com/paper/2605.20797/) (2026)

## Abstract (arXiv)

Benchmarking is a key aspect of research into optimization algorithms, and as such the way in which the most popular benchmark suites are designed implicitly guides some parts of algorithm design. One of these suites is the black-box optimization benchmarking (BBOB) suite of 24 single-objective noiseless functions, which has been a standard for over a decade. Within this problem suite, different instances of a single problem can be created, which is beneficial for testing the stability and invariance of algorithms under transformations. In this paper, we investigate the BBOB instance creation protocol by considering a set of 500 instances for each BBOB problem. Using exploratory landscape analysis, we show that the distribution of landscape features across BBOB instances is highly diverse for a large set of problems. In addition, we run a set of eight algorithms across these 500 instances, and investigate for which cases statistically significant differences in performance occur. We argue that, while the transformations applied in BBOB instances do indeed seem to preserve the high-level properties of the functions, their difference in practice should not be overlooked, particularly when treating the problems as box-constrained instead of unconstrained.