# COCO: Performance Assessment

arXiv: https://arxiv.org/abs/1605.03560 · HTML: https://mlsanity.com/paper/1605.03560/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2016.

## Built on by (most cited first, 8 of 8)

- [Understanding Impacts of Electromagnetic Signal Injection Attacks on Object Detection](https://mlsanity.com/paper/2407.16327/) (2024)
- [A Tutorial on the Design, Experimentation and Application of Metaheuristic Algorithms to Real-World Optimization Problems](https://mlsanity.com/paper/2410.03205/) (2024)
- [AdaFPP: Adapt-Focused Bi-Propagating Prototype Learning for Panoramic Activity Recognition](https://mlsanity.com/paper/2405.02538/) (2024)
- [MicroNAS: An Automated Framework for Developing a Fall Detection System](https://mlsanity.com/paper/2504.07397/) (2025)
- [Trial-Based Dominance Enables Non-Parametric Tests to Compare both the Speed and Accuracy of Stochastic Optimizers](https://mlsanity.com/paper/2212.09423/) (2022)
- [Better Understandings and Configurations in MaxSAT Local Search Solvers via Anytime Performance Analysis](https://mlsanity.com/paper/2403.06568/) (2024)
- [Geometric Probing for Algorithm Selection in Continuous Black-Box Optimization](https://mlsanity.com/paper/2604.09095/) (2026)
- [What a Reporting Convention Hides: A Matched-Budget Audit of Quantum Natural Gradient with an Exactly Computed Metric](https://mlsanity.com/paper/2610.09425/) (2026)

## Abstract (arXiv)

We present an any-time performance assessment for benchmarking numerical optimization algorithms in a black-box scenario, applied within the COCO benchmarking platform. The performance assessment is based on runtimes measured in number of objective function evaluations to reach one or several quality indicator target values. We argue that runtime is the only available measure with a generic, meaningful, and quantitative interpretation. We discuss the choice of the target values, runlength-based targets, and the aggregation of results by using simulated restarts, averages, and empirical distribution functions.