# Optimizing with Low Budgets: a Comparison on the Black-box Optimization Benchmarking Suite and OpenAI Gym

arXiv: https://arxiv.org/abs/2310.00077 · HTML: https://mlsanity.com/paper/2310.00077/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 92 references that are in the map.
- First posted on arXiv 29 Sep 2023.

## Builds on (most cited first, 28 of 28)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Bayesian Optimization in a Billion Dimensions via Random Embeddings](https://mlsanity.com/paper/1301.1942/) (2013)
- [Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020](https://mlsanity.com/paper/2104.10201/) (2021)
- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [Batched Large-scale Bayesian Optimization in High-dimensional Spaces](https://mlsanity.com/paper/1706.01445/) (2017)
- [Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search](https://mlsanity.com/paper/2007.00708/) (2020)
- [NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search](https://mlsanity.com/paper/2001.10422/) (2020)
- [D2RL: Deep Dense Architectures in Reinforcement Learning](https://mlsanity.com/paper/2010.09163/) (2020)
- [Benchmarking in Optimization: Best Practice and Open Issues](https://mlsanity.com/paper/2007.03488/) (2020)
- [ASlib: A Benchmark Library for Algorithm Selection](https://mlsanity.com/paper/1506.02465/) (2015)
- [NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy](https://mlsanity.com/paper/2201.13396/) (2022)
- [IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics](https://mlsanity.com/paper/1810.05281/) (2018)
- [IOHanalyzer: Detailed Performance Analyses for Iterative Optimization Heuristics](https://mlsanity.com/paper/2007.03953/) (2020)
- [Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systems](https://mlsanity.com/paper/2103.07932/) (2021)
- [Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking](https://mlsanity.com/paper/2010.04542/) (2020)
- [Reproducibility in Evolutionary Computation](https://mlsanity.com/paper/2102.03380/) (2021)
- [Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers](https://mlsanity.com/paper/1804.00154/) (2018)
- [EXPObench: Benchmarking Surrogate-based Optimisation Algorithms on Expensive Black-box Functions](https://mlsanity.com/paper/2106.04618/) (2021)
- [Fully Parallel Hyperparameter Search: Reshaped Space-Filling](https://mlsanity.com/paper/1910.08406/) (2019)
- [Continuous Multi-objective Zero-touch Network Slicing via Twin Delayed DDPG and OpenAI Gym](https://mlsanity.com/paper/2101.06617/) (2021)
- [How Much Do Unstated Problem Constraints Limit Deep Robotic Reinforcement Learning?](https://mlsanity.com/paper/1909.09282/) (2019)
- [Meta-Model Framework for Surrogate-Based Parameter Estimation in Dynamical Systems](https://mlsanity.com/paper/1906.09088/) (2019)
- [The False Dawn: Reevaluating Google's Reinforcement Learning for Chip Macro Placement](https://mlsanity.com/paper/2306.09633/) (2023)
- [Variance Reduction for Better Sampling in Continuous Domains](https://mlsanity.com/paper/2004.11687/) (2020)
- [Surrogate Models for Optimization of Dynamical Systems](https://mlsanity.com/paper/2101.10189/) (2021)

## Built on by (most cited first, 4 of 4)

- [Performance Comparison of Surrogate-Assisted Evolutionary Algorithms on Computational Fluid Dynamics Problems](https://mlsanity.com/paper/2402.16455/) (2024)
- [Bellman Memory Units: A neuromorphic framework for synaptic reinforcement learning with an evolving network topology](https://mlsanity.com/paper/2511.16066/) (2025)
- [Empirical Evaluation of No Free Lunch Violations in Permutation-Based Optimization](https://mlsanity.com/paper/2603.03613/) (2026)
- [MMAO-Cls: Metabolic Multi-Agent Optimization for Joint Feature Selection and Classifier Tuning](https://mlsanity.com/paper/2607.01539/) (2026)

## Abstract (arXiv)

The growing ubiquity of machine learning (ML) has led it to enter various areas of computer science, including black-box optimization (BBO). Recent research is particularly concerned with Bayesian optimization (BO). BO-based algorithms are popular in the ML community, as they are used for hyperparameter optimization and more generally for algorithm configuration. However, their efficiency decreases as the dimensionality of the problem and the budget of evaluations increase. Meanwhile, derivative-free optimization methods have evolved independently in the optimization community. Therefore, we urge to understand whether cross-fertilization is possible between the two communities, ML and BBO, i.e., whether algorithms that are heavily used in ML also work well in BBO and vice versa. Comparative experiments often involve rather small benchmarks and show visible problems in the experimental setup, such as poor initialization of baselines, overfitting due to problem-specific setting of hyperparameters, and low statistical significance. With this paper, we update and extend a comparative study presented by Hutter et al. in 2013. We compare BBO tools for ML with more classical heuristics, first on the well-known BBOB benchmark suite from the COCO environment and then on Direct Policy Search for OpenAI Gym, a reinforcement learning benchmark. Our results confirm that BO-based optimizers perform well on both benchmarks when budgets are limited, albeit with a higher computational cost, while they are often outperformed by algorithms from other families when the evaluation budget becomes larger. We also show that some algorithms from the BBO community perform surprisingly well on ML tasks.