# BONO-Bench: A Comprehensive Test Suite for Bi-objective Numerical Optimization with Traceable Pareto Sets

arXiv: https://arxiv.org/abs/2601.16970 · HTML: https://mlsanity.com/paper/2601.16970/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 51 references that are in the map.
- First posted on arXiv 23 Jan 2026.

## Builds on (most cited first, 11 of 11)

- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [pymoo: Multi-objective Optimization in Python](https://mlsanity.com/paper/2002.04504/) (2020)
- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [Automated Algorithm Selection: Survey and Perspectives](https://mlsanity.com/paper/1811.11597/) (2018)
- [The Hypervolume Indicator: Problems and Algorithms](https://mlsanity.com/paper/2005.00515/) (2020)
- [Using Well-Understood Single-Objective Functions in Multiobjective Black-Box Optimization Test Suites](https://mlsanity.com/paper/1604.00359/) (2016)
- [One PLOT to Show Them All: Visualization of Efficient Sets in Multi-Objective Landscapes](https://mlsanity.com/paper/2006.11547/) (2020)
- [Challenges of Convex Quadratic Bi-objective Benchmark Problems](https://mlsanity.com/paper/1810.09690/) (2018)
- [On Bi-Objective convex-quadratic problems](https://mlsanity.com/paper/1812.00289/) (2018)
- [MOLE: Digging Tunnels Through Multimodal Multi-Objective Landscapes](https://mlsanity.com/paper/2204.10848/) (2022)
- [R2 v2: The Pareto-compliant R2 Indicator for Better Benchmarking in Bi-objective Optimization](https://mlsanity.com/paper/2407.01504/) (2024)

## Abstract (arXiv)

The evaluation of heuristic optimizers on test problems, better known as benchmarking, is a cornerstone of research in multi-objective optimization. However, most test problems used in benchmarking numerical multi-objective black-box optimizers come from one of two flawed approaches: On the one hand, problems are constructed manually, which result in problems with well-understood optimal solutions, but unrealistic properties and biases. On the other hand, more realistic and complex single-objective problems are composited into multi-objective problems, but with a lack of control and understanding of problem properties. This paper proposes an extensive problem generation approach for bi-objective numerical optimization problems consisting of the combination of theoretically well-understood convex-quadratic functions into unimodal and multimodal landscapes with and without global structure. It supports configuration of test problem properties, such as the number of decision variables, local optima, Pareto front shape, plateaus in the objective space, or degree of conditioning, while maintaining theoretical tractability: The optimal front can be approximated to an arbitrary degree of precision regarding Pareto-compliant performance indicators such as the hypervolume or the exact R2 indicator. To demonstrate the generator's capabilities, a test suite of 20 problem categories, called BONO-Bench, is created and subsequently used as a basis of an illustrative benchmark study. Finally, the general approach underlying our proposed generator, together with the associated test suite, is publicly released in the Python package textttbonobench to facilitate reproducible benchmarking.