# IOHexperimenter: Benchmarking Platform for Iterative Optimization Heuristics

arXiv: https://arxiv.org/abs/2111.04077 · HTML: https://mlsanity.com/paper/2111.04077/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2021.

## Built on by (most cited first, 24 of 24)

- [LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics](https://mlsanity.com/paper/2405.20132/) (2024)
- [A Systematic Survey on Large Language Models for Algorithm Design](https://mlsanity.com/paper/2410.14716/) (2024)
- [MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning](https://mlsanity.com/paper/2310.08252/) (2023)
- [In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics](https://mlsanity.com/paper/2410.16309/) (2024)
- [LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithms](https://mlsanity.com/paper/2505.21034/) (2025)
- [To Switch or not to Switch: Predicting the Benefit of Switching between Algorithms based on Trajectory Features](https://mlsanity.com/paper/2302.09075/) (2023)
- [MA-BBOB: Many-Affine Combinations of BBOB Functions for Evaluating AutoML Approaches in Noiseless Numerical Black-Box Optimization Contexts](https://mlsanity.com/paper/2306.10627/) (2023)
- [Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systems](https://mlsanity.com/paper/2505.15741/) (2025)
- [Generalization Ability of Feature-based Performance Prediction Models: A Statistical Analysis across Benchmarks](https://mlsanity.com/paper/2405.12259/) (2024)
- [Behaviour Space Analysis of LLM-driven Meta-heuristic Discovery](https://mlsanity.com/paper/2507.03605/) (2025)
- [Bencher: Simple and Reproducible Benchmarking for Black-Box Optimization](https://mlsanity.com/paper/2505.21321/) (2025)
- [The importance of being constrained: dealing with infeasible solutions in Differential Evolution and beyond](https://mlsanity.com/paper/2203.03512/) (2022)
- [Benchmarking Algorithms for Submodular Optimization Problems Using IOHProfiler](https://mlsanity.com/paper/2302.01464/) (2023)
- [Transfer Learning of Surrogate Models via Domain Affine Transformation Across Synthetic and Real-World Benchmarks](https://mlsanity.com/paper/2501.14012/) (2025)
- [Comparing Optimization Algorithms Through the Lens of Search Behavior Analysis](https://mlsanity.com/paper/2507.01668/) (2025)
- [Better Understandings and Configurations in MaxSAT Local Search Solvers via Anytime Performance Analysis](https://mlsanity.com/paper/2403.06568/) (2024)
- [A Deep Dive into Effects of Structural Bias on CMA-ES Performance along Affine Trajectories](https://mlsanity.com/paper/2404.17323/) (2024)
- [A Novel Ranking Scheme for the Performance Analysis of Stochastic Optimization Algorithms using the Principles of Severity](https://mlsanity.com/paper/2406.00154/) (2024)
- [Transfer Learning of Surrogate Models: Integrating Domain Warping and Affine Transformations](https://mlsanity.com/paper/2501.18344/) (2025)
- [Bayesian Optimization by Kernel Regression and Density-based Exploration](https://mlsanity.com/paper/2502.06178/) (2025)
- [SEvoBench : A C++ Framework For Evolutionary Single-Objective Optimization Benchmarking](https://mlsanity.com/paper/2505.17430/) (2025)
- [From Performance to Understanding: A Vision for Explainable Automated Algorithm Design](https://mlsanity.com/paper/2511.16201/) (2025)
- [Does Dimensionality Reduction via Random Projections Preserve Landscape Features?](https://mlsanity.com/paper/2604.13230/) (2026)
- [On the Influence of the Feature Computation Budget on Per-Instance Algorithm Selection for Black-Box Optimization](https://mlsanity.com/paper/2605.04954/) (2026)

## Abstract (arXiv)

We present IOHexperimenter, the experimentation module of the IOHprofiler project, which aims at providing an easy-to-use and highly customizable toolbox for benchmarking iterative optimization heuristics such as local search, evolutionary and genetic algorithms, Bayesian optimization techniques, etc. IOHexperimenter can be used as a stand-alone tool or as part of a benchmarking pipeline that uses other components of IOHprofiler such as IOHanalyzer, the module for interactive performance analysis and visualization. IOHexperimenter provides an efficient interface between optimization problems and their solvers while allowing for granular logging of the optimization process. These logs are fully compatible with existing tools for interactive data analysis, which significantly speeds up the deployment of a benchmarking pipeline. The main components of IOHexperimenter are the environment to build customized problem suites and the various logging options that allow users to steer the granularity of the data records.