# OPTION: OPTImization Algorithm Benchmarking ONtology

arXiv: https://arxiv.org/abs/2211.11332 · HTML: https://mlsanity.com/paper/2211.11332/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 63 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [Automated Algorithm Selection: Survey and Perspectives](https://mlsanity.com/paper/1811.11597/) (2018)
- [Benchmarking in Optimization: Best Practice and Open Issues](https://mlsanity.com/paper/2007.03488/) (2020)
- [IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics](https://mlsanity.com/paper/1810.05281/) (2018)
- [Comprehensive Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems Using the R-Package flacco](https://mlsanity.com/paper/1708.05258/) (2017)
- [Exploratory Landscape Analysis is Strongly Sensitive to the Sampling Strategy](https://mlsanity.com/paper/2006.11135/) (2020)
- [Versatile Black-Box Optimization](https://mlsanity.com/paper/2004.14014/) (2020)
- [IOHanalyzer: Detailed Performance Analyses for Iterative Optimization Heuristics](https://mlsanity.com/paper/2007.03953/) (2020)
- [Towards Feature-Based Performance Regression Using Trajectory Data](https://mlsanity.com/paper/2102.05370/) (2021)
- [Benchmarking Discrete Optimization Heuristics with IOHprofiler](https://mlsanity.com/paper/1912.09237/) (2019)
- [Reproducibility in Evolutionary Computation](https://mlsanity.com/paper/2102.03380/) (2021)
- [Personalizing Performance Regression Models to Black-Box Optimization Problems](https://mlsanity.com/paper/2104.10999/) (2021)

## Abstract (arXiv)

Many optimization algorithm benchmarking platforms allow users to share their experimental data to promote reproducible and reusable research. However, different platforms use different data models and formats, which drastically complicates the identification of relevant datasets, their interpretation, and their interoperability. Therefore, a semantically rich, ontology-based, machine-readable data model that can be used by different platforms is highly desirable. In this paper, we report on the development of such an ontology, which we call OPTION (OPTImization algorithm benchmarking ONtology). Our ontology provides the vocabulary needed for semantic annotation of the core entities involved in the benchmarking process, such as algorithms, problems, and evaluation measures. It also provides means for automatic data integration, improved interoperability, and powerful querying capabilities, thereby increasing the value of the benchmarking data. We demonstrate the utility of OPTION, by annotating and querying a corpus of benchmark performance data from the BBOB collection of the COCO framework and from the Yet Another Black-Box Optimization Benchmark (YABBOB) family of the Nevergrad environment. In addition, we integrate features of the BBOB functional performance landscape into the OPTION knowledge base using publicly available datasets with exploratory landscape analysis. Finally, we integrate the OPTION knowledge base into the IOHprofiler environment and provide users with the ability to perform meta-analysis of performance data.