# Large-scale Benchmarking of Metaphor-based Optimization Heuristics

arXiv: https://arxiv.org/abs/2402.09800 · HTML: https://mlsanity.com/paper/2402.09800/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2024.

## Built on by (most cited first, 8 of 8)

- [LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics](https://mlsanity.com/paper/2405.20132/) (2024)
- [OptunaHub: A Platform for Black-Box Optimization](https://mlsanity.com/paper/2510.02798/) (2025)
- [Comparing Optimization Algorithms Through the Lens of Search Behavior Analysis](https://mlsanity.com/paper/2507.01668/) (2025)
- [Advancements in Multimodal Differential Evolution: A Comprehensive Review and Future Perspectives](https://mlsanity.com/paper/2504.00717/) (2025)
- [A Review on Single-Problem Multi-Attempt Heuristic Optimization](https://mlsanity.com/paper/2509.26321/) (2025)
- [From Performance to Understanding: A Vision for Explainable Automated Algorithm Design](https://mlsanity.com/paper/2511.16201/) (2025)
- [Best Practices For Empirical Meta-Algorithmic Research: Guidelines from the COSEAL Research Network](https://mlsanity.com/paper/2512.16491/) (2025)
- [MMAO-Cls: Metabolic Multi-Agent Optimization for Joint Feature Selection and Classifier Tuning](https://mlsanity.com/paper/2607.01539/) (2026)

## Abstract (arXiv)

The number of proposed iterative optimization heuristics is growing steadily, and with this growth, there have been many points of discussion within the wider community. One particular criticism that is raised towards many new algorithms is their focus on metaphors used to present the method, rather than emphasizing their potential algorithmic contributions. Several studies into popular metaphor-based algorithms have highlighted these problems, even showcasing algorithms that are functionally equivalent to older existing methods. Unfortunately, this detailed approach is not scalable to the whole set of metaphor-based algorithms. Because of this, we investigate ways in which benchmarking can shed light on these algorithms. To this end, we run a set of 294 algorithm implementations on the BBOB function suite. We investigate how the choice of the budget, the performance measure, or other aspects of experimental design impact the comparison of these algorithms. Our results emphasize why benchmarking is a key step in expanding our understanding of the algorithm space, and what challenges still need to be overcome to fully gauge the potential improvements to the state-of-the-art hiding behind the metaphors.