# A Collection of Quality Diversity Optimization Problems Derived from Hyperparameter Optimization of Machine Learning Models

arXiv: https://arxiv.org/abs/2204.14061 · HTML: https://mlsanity.com/paper/2204.14061/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 28 Apr 2022.

## Builds on (most cited first, 12 of 12)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space](https://mlsanity.com/paper/1912.02400/) (2019)
- [Differentiable Quality Diversity](https://mlsanity.com/paper/2106.03894/) (2021)
- [YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization](https://mlsanity.com/paper/2109.03670/) (2021)
- [Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability](https://mlsanity.com/paper/1904.03867/) (2019)
- [BOP-Elites, a Bayesian Optimisation algorithm for Quality-Diversity search](https://mlsanity.com/paper/2005.04320/) (2020)
- [Pruning Random Forests for Prediction on a Budget](https://mlsanity.com/paper/1606.05060/) (2016)
- [Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters](https://mlsanity.com/paper/2007.05352/) (2020)

## Built on by (most cited first, 3 of 3)

- [pyribs: A Bare-Bones Python Library for Quality Diversity Optimization](https://mlsanity.com/paper/2303.00191/) (2023)
- [Evolutionary Reinforcement Learning: A Survey](https://mlsanity.com/paper/2303.04150/) (2023)
- [Tackling Neural Architecture Search With Quality Diversity Optimization](https://mlsanity.com/paper/2208.00204/) (2022)

## Abstract (arXiv)

The goal of Quality Diversity Optimization is to generate a collection of diverse yet high-performing solutions to a given problem at hand. Typical benchmark problems are, for example, finding a repertoire of robot arm configurations or a collection of game playing strategies. In this paper, we propose a set of Quality Diversity Optimization problems that tackle hyperparameter optimization of machine learning models - a so far underexplored application of Quality Diversity Optimization. Our benchmark problems involve novel feature functions, such as interpretability or resource usage of models. To allow for fast and efficient benchmarking, we build upon YAHPO Gym, a recently proposed open source benchmarking suite for hyperparameter optimization that makes use of high performing surrogate models and returns these surrogate model predictions instead of evaluating the true expensive black box function. We present results of an initial experimental study comparing different Quality Diversity optimizers on our benchmark problems. Furthermore, we discuss future directions and challenges of Quality Diversity Optimization in the context of hyperparameter optimization.