mlsanityData to 7 Oct 2026

Explainable Benchmarking for Iterative Optimization Heuristics

arXiv 2401.17842 ↗First posted 31 Jan 2024cs.NE cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Benchmarking heuristic algorithms is vital to understand under which conditions and on what kind of problems certain algorithms perform well. In most current research into heuristic optimization algorithms, only a very limited number of scenarios, algorithm configurations and hyper-parameter settings are explored, leading to incomplete and often biased insights and results. This paper presents a novel approach we call explainable benchmarking. Introducing the IOH-Xplainer software framework, for analyzing and understanding the performance of various optimization algorithms and the impact of their different components and hyper-parameters. We showcase the framework in the context of two modular optimization frameworks. Through this framework, we examine the impact of different algorithmic components and configurations, offering insights into their performance across diverse scenarios. We provide a systematic method for evaluating and interpreting the behaviour and efficiency of iterative optimization heuristics in a more transparent and comprehensible manner, allowing for better benchmarking and algorithm design.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 28 of its 80 references that are in the map. First posted on arXiv 31 Jan 2024.

Builds on

28 of 80 references.

References · 28

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 229
  4. 2016COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Settingcited by 135
  5. 2020An Easy-to-use Real-world Multi-objective Optimization Problem Suitecited by 71
  6. 2020Benchmarking in Optimization: Best Practice and Open Issuescited by 35
  7. 2017Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learningcited by 33
  8. 2022A Survey of Methods for Automated Algorithm Configurationcited by 29
  9. 2019BOAH: A Tool Suite for Multi-Fidelity Bayesian Optimization & Analysis of Hyperparameterscited by 25
  10. 2022Covariance Matrix Adaptation MAP-Annealingcited by 24
  11. 2020IOHanalyzer: Detailed Performance Analyses for Iterative Optimization Heuristicscited by 20
  12. 2021Tuning as a Means of Assessing the Benefits of New Ideas in Interplay with Existing Algorithmic Modulescited by 20
16 more
  1. 2023DoE2Vec: Deep-learning Based Features for Exploratory Landscape Analysiscited by 12
  2. 2023Modular Differential Evolutioncited by 11
  3. 2020Applying Genetic Programming to Improve Interpretability in Machine Learning Modelscited by 10
  4. 2022BBOB Instance Analysis: Landscape Properties and Algorithm Performance across Problem Instancescited by 10
  5. 2014Structural bias in population-based algorithmscited by 9
  6. 2020Genetic Programming for Evolving a Front of Interpretable Models for Data Visualisationcited by 9
  7. 2020Identifying Properties of Real-World Optimisation Problems through a Questionnairecited by 5
  8. 2021Explainable Landscape-Aware Optimization Performance Predictioncited by 5
  9. 2022The Importance of Landscape Features for Performance Prediction of Modular CMA-ES Variantscited by 5
  10. 2021Paradiseo: From a Modular Framework for Evolutionary Computation to the Automated Design of Metaheuristics ---22 Years of Paradiseo---cited by 4
  11. 2022Improving Nevergrad's Algorithm Selection Wizard NGOpt through Automated Algorithm Configurationcited by 4
  12. 2023Using Knowledge Graphs for Performance Prediction of Modular Optimization Algorithmscited by 4
  13. 2023Deep-BIAS: Detecting Structural Bias using Explainable AIcited by 4
  14. 2021Quantifying the Impact of Boundary Constraint Handling Methods on Differential Evolutioncited by 3
  15. 2022The importance of being constrained: dealing with infeasible solutions in Differential Evolution and beyondcited by 1
  16. 2023Patterns of Convergence and Bound Constraint Violation in Differential Evolution on SBOX-COST Benchmarking Suitecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 3.

10 papers.

  1. 2024LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristicscited by 92
  2. 2025BLADE: Benchmark suite for LLM-driven Automated Design and Evolution of iterative optimisation heuristicscited by 9
  3. 2024A Survey of Features Used for Representing Black-box Single-objective Continuous Optimizationcited by 3
  4. 2024Evolutionary Computation and Explainable AI: A Roadmap to Understandable Intelligent Systemscited by 2
  5. 2024Landscape-Aware Automated Algorithm Configuration using Multi-output Mixed Regression and Classificationcited by 1
  6. 2025Tracing the Interactions of Modular CMA-ES Configurations Across Problem Landscapescited by 1
  7. 2025From Performance to Understanding: A Vision for Explainable Automated Algorithm Designcited by 0
  8. 2026Quantifying the Impact of Modules and Their Interactions in the PSO-X Frameworkcited by 0
  9. 2026Pareto-Optimal Anytime Algorithms via Bayesian Racingcited by 0
  10. 2026Performance and Explainability Requirements of Evolutionary Algorithms in Real-World Physics-Informed Optimizationcited by 0