# Using the Empirical Attainment Function for Analyzing Single-objective Black-box Optimization Algorithms

arXiv: https://arxiv.org/abs/2404.02031 · HTML: https://mlsanity.com/paper/2404.02031/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2024.

## Built on by (most cited first, 6 of 6)

- [LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics](https://mlsanity.com/paper/2405.20132/) (2024)
- [In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics](https://mlsanity.com/paper/2410.16309/) (2024)
- [MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization](https://mlsanity.com/paper/2505.17745/) (2025)
- [Explainable Information Processing in Particle Swarm Optimization through Landscape and Search Behavior Analysis](https://mlsanity.com/paper/2509.06272/) (2025)
- [Geometric Probing for Algorithm Selection in Continuous Black-Box Optimization](https://mlsanity.com/paper/2604.09095/) (2026)
- [On the Influence of the Feature Computation Budget on Per-Instance Algorithm Selection for Black-Box Optimization](https://mlsanity.com/paper/2605.04954/) (2026)

## Abstract (arXiv)

A widely accepted way to assess the performance of iterative black-box optimizers is to analyze their empirical cumulative distribution function (ECDF) of pre-defined quality targets achieved not later than a given runtime. In this work, we consider an alternative approach, based on the empirical attainment function (EAF) and we show that the target-based ECDF is an approximation of the EAF. We argue that the EAF has several advantages over the target-based ECDF. In particular, it does not require defining a priori quality targets per function, captures performance differences more precisely, and enables the use of additional summary statistics that enrich the analysis. We also show that the average area over the convergence curves is a simpler-to-calculate, but equivalent, measure of anytime performance. To facilitate the accessibility of the EAF, we integrate a module to compute it into the IOHanalyzer platform. Finally, we illustrate the use of the EAF via synthetic examples and via the data available for the BBOB suite.