# Adaptive Estimation of the Number of Algorithm Runs in Stochastic Optimization

arXiv: https://arxiv.org/abs/2507.01629 · HTML: https://mlsanity.com/paper/2507.01629/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2025.

## Built on by (most cited first, 2 of 2)

- [Pareto-Optimal Anytime Algorithms via Bayesian Racing](https://mlsanity.com/paper/2603.08493/) (2026)
- [Learning to Assess the Reliability of Number-of-Runs Estimation in Stochastic Optimization](https://mlsanity.com/paper/2605.28309/) (2026)

## Abstract (arXiv)

Determining the number of algorithm runs is a critical aspect of experimental design, as it directly influences the experiment's duration and the reliability of its outcomes. This paper introduces an empirical approach to estimating the required number of runs per problem instance for accurate estimation of the performance of the continuous single-objective stochastic optimization algorithm. The method leverages probability theory, incorporating a robustness check to identify significant imbalances in the data distribution relative to the mean, and dynamically adjusts the number of runs during execution as an online approach. The proposed methodology was extensively tested across two algorithm portfolios (104 Differential Evolution configurations and the Nevergrad portfolio) and the COCO benchmark suite, totaling 5748000 runs. The results demonstrate 82% - 95% accuracy in estimations across different algorithms, allowing a reduction of approximately 50% in the number of runs without compromising optimization outcomes. This online calculation of required runs not only improves benchmarking efficiency, but also contributes to energy reduction, fostering a more environmentally sustainable computing ecosystem.