# Tuning as a Means of Assessing the Benefits of New Ideas in Interplay with Existing Algorithmic Modules

arXiv: https://arxiv.org/abs/2102.12905 · HTML: https://mlsanity.com/paper/2102.12905/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2021.

## Built on by (most cited first, 20 of 20)

- [LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics](https://mlsanity.com/paper/2405.20132/) (2024)
- [Explainable Benchmarking for Iterative Optimization Heuristics](https://mlsanity.com/paper/2401.17842/) (2024)
- [To Switch or not to Switch: Predicting the Benefit of Switching between Algorithms based on Trajectory Features](https://mlsanity.com/paper/2302.09075/) (2023)
- [BLADE: Benchmark suite for LLM-driven Automated Design and Evolution of iterative optimisation heuristics](https://mlsanity.com/paper/2504.20183/) (2025)
- [MA-BBOB: Many-Affine Combinations of BBOB Functions for Evaluating AutoML Approaches in Noiseless Numerical Black-Box Optimization Contexts](https://mlsanity.com/paper/2306.10627/) (2023)
- [Explainable Landscape-Aware Optimization Performance Prediction](https://mlsanity.com/paper/2110.11633/) (2021)
- [The Importance of Landscape Features for Performance Prediction of Modular CMA-ES Variants](https://mlsanity.com/paper/2204.07431/) (2022)
- [PS-AAS: Portfolio Selection for Automated Algorithm Selection in Black-Box Optimization](https://mlsanity.com/paper/2310.10685/) (2023)
- [Automated Benchmark-Driven Design and Explanation of Hyperparameter Optimizers](https://mlsanity.com/paper/2111.14756/) (2021)
- [Using Knowledge Graphs for Performance Prediction of Modular Optimization Algorithms](https://mlsanity.com/paper/2301.09876/) (2023)
- [Behaviour Space Analysis of LLM-driven Meta-heuristic Discovery](https://mlsanity.com/paper/2507.03605/) (2025)
- [Explainable Landscape Analysis in Automated Algorithm Performance Prediction](https://mlsanity.com/paper/2203.11828/) (2022)
- [A Survey of Features Used for Representing Black-box Single-objective Continuous Optimization](https://mlsanity.com/paper/2406.06629/) (2024)
- [The importance of being constrained: dealing with infeasible solutions in Differential Evolution and beyond](https://mlsanity.com/paper/2203.03512/) (2022)
- [Landscape-Aware Automated Algorithm Configuration using Multi-output Mixed Regression and Classification](https://mlsanity.com/paper/2409.01446/) (2024)
- [METAFOR: A Hybrid Metaheuristics Software Framework for Single-Objective Continuous Optimization Problems](https://mlsanity.com/paper/2502.11225/) (2025)
- [Tracing the Interactions of Modular CMA-ES Configurations Across Problem Landscapes](https://mlsanity.com/paper/2507.02331/) (2025)
- [A Deep Dive into Effects of Structural Bias on CMA-ES Performance along Affine Trajectories](https://mlsanity.com/paper/2404.17323/) (2024)
- [Quantifying the Impact of Modules and Their Interactions in the PSO-X Framework](https://mlsanity.com/paper/2601.04100/) (2026)
- [Pareto-Optimal Anytime Algorithms via Bayesian Racing](https://mlsanity.com/paper/2603.08493/) (2026)

## Abstract (arXiv)

Introducing new algorithmic ideas is a key part of the continuous improvement of existing optimization algorithms. However, when introducing a new component into an existing algorithm, assessing its potential benefits is a challenging task. Often, the component is added to a default implementation of the underlying algorithm and compared against a limited set of other variants. This assessment ignores any potential interplay with other algorithmic ideas that share the same base algorithm, which is critical in understanding the exact contributions being made. We introduce a more extensive procedure, which uses hyperparameter tuning as a means of assessing the benefits of new algorithmic components. This allows for a more robust analysis by not only focusing on the impact on performance, but also by investigating how this performance is achieved. We implement our suggestion in the context of the Modular CMA-ES framework, which was redesigned and extended to include some new modules and several new options for existing modules, mostly focused on the step-size adaptation method. Our analysis highlights the differences between these new modules, and identifies the situations in which they have the largest contribution.