# DACBench: A Benchmark Library for Dynamic Algorithm Configuration

arXiv: https://arxiv.org/abs/2105.08541 · HTML: https://mlsanity.com/paper/2105.08541/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 18 May 2021.

## Builds on (most cited first, 14 of 14)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [Contextual Markov Decision Processes](https://mlsanity.com/paper/1502.02259/) (2015)
- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [Meta-Reinforcement Learning of Structured Exploration Strategies](https://mlsanity.com/paper/1802.07245/) (2018)
- [ParamILS: An Automatic Algorithm Configuration Framework](https://mlsanity.com/paper/1401.3492/) (2014)
- [Evolved Policy Gradients](https://mlsanity.com/paper/1802.04821/) (2018)
- [ASlib: A Benchmark Library for Algorithm Selection](https://mlsanity.com/paper/1506.02465/) (2015)
- [IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics](https://mlsanity.com/paper/1810.05281/) (2018)
- [Learning Heuristic Selection with Dynamic Algorithm Configuration](https://mlsanity.com/paper/2006.08246/) (2020)
- [Online Selection of CMA-ES Variants](https://mlsanity.com/paper/1904.07801/) (2019)
- [The Configurable SAT Solver Challenge (CSSC)](https://mlsanity.com/paper/1505.01221/) (2015)

## Built on by (most cited first, 28 of 28)

- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Automated Dynamic Algorithm Configuration](https://mlsanity.com/paper/2205.13881/) (2022)
- [Multi-agent Dynamic Algorithm Configuration](https://mlsanity.com/paper/2210.06835/) (2022)
- [A Survey of Methods for Automated Algorithm Configuration](https://mlsanity.com/paper/2202.01651/) (2022)
- [ConfigX: Modular Configuration for Evolutionary Algorithms via Multitask Reinforcement Learning](https://mlsanity.com/paper/2412.07507/) (2024)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning](https://mlsanity.com/paper/2502.19652/) (2025)
- [Theory-inspired Parameter Control Benchmarks for Dynamic Algorithm Configuration](https://mlsanity.com/paper/2202.03259/) (2022)
- [EXPObench: Benchmarking Surrogate-based Optimisation Algorithms on Expensive Black-box Functions](https://mlsanity.com/paper/2106.04618/) (2021)
- [To Switch or not to Switch: Predicting the Benefit of Switching between Algorithms based on Trajectory Features](https://mlsanity.com/paper/2302.09075/) (2023)
- [Hierarchical Transformers are Efficient Meta-Reinforcement Learners](https://mlsanity.com/paper/2402.06402/) (2024)
- [Can we hop in general? A discussion of benchmark selection and design using the Hopper environment](https://mlsanity.com/paper/2410.08870/) (2024)
- [Using Automated Algorithm Configuration for Parameter Control](https://mlsanity.com/paper/2302.12334/) (2023)
- [Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization](https://mlsanity.com/paper/2407.13513/) (2024)
- [CALE: Continuous Arcade Learning Environment](https://mlsanity.com/paper/2410.23810/) (2024)
- [On the Importance of Reward Design in Reinforcement Learning-based Dynamic Algorithm Configuration: A Case Study on OneMax with (1+(λ,λ))-GA](https://mlsanity.com/paper/2502.20265/) (2025)
- [CANDID DAC: Leveraging Coupled Action Dimensions with Importance Differences in DAC](https://mlsanity.com/paper/2407.05789/) (2024)
- [Sequential Multi-Agent Dynamic Algorithm Configuration](https://mlsanity.com/paper/2510.23535/) (2025)
- [Automated Machine Learning, Bounded Rationality, and Rational Metareasoning](https://mlsanity.com/paper/2109.04744/) (2021)
- [Discovering Multiple Algorithm Configurations](https://mlsanity.com/paper/2303.07434/) (2023)
- [Deep Reinforcement Learning for Dynamic Algorithm Configuration: A Case Study on Optimizing OneMax with the (1+(λ,λ))-GA](https://mlsanity.com/paper/2512.03805/) (2025)
- [Large Language Models as Optimization Controllers: Adaptive Continuation for SIMP Topology Optimization](https://mlsanity.com/paper/2603.25099/) (2026)
- [Explainable Model-specific Algorithm Selection for Multi-Label Classification](https://mlsanity.com/paper/2211.11227/) (2022)
- [A Research Agenda for Usability and Generalisation in Reinforcement Learning](https://mlsanity.com/paper/2412.16970/) (2024)
- [Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization](https://mlsanity.com/paper/2505.16471/) (2025)
- [Revisiting Learning Rate Control](https://mlsanity.com/paper/2507.01724/) (2025)
- [OPAL: Operator-Programmed Algorithms for Landscape-Aware Black-Box Optimization](https://mlsanity.com/paper/2512.12809/) (2025)
- [Best Practices For Empirical Meta-Algorithmic Research: Guidelines from the COSEAL Research Network](https://mlsanity.com/paper/2512.16491/) (2025)

## Abstract (arXiv)

Dynamic Algorithm Configuration (DAC) aims to dynamically control a target algorithm's hyperparameters in order to improve its performance. Several theoretical and empirical results have demonstrated the benefits of dynamically controlling hyperparameters in domains like evolutionary computation, AI Planning or deep learning. Replicating these results, as well as studying new methods for DAC, however, is difficult since existing benchmarks are often specialized and incompatible with the same interfaces. To facilitate benchmarking and thus research on DAC, we propose DACBench, a benchmark library that seeks to collect and standardize existing DAC benchmarks from different AI domains, as well as provide a template for new ones. For the design of DACBench, we focused on important desiderata, such as (i) flexibility, (ii) reproducibility, (iii) extensibility and (iv) automatic documentation and visualization. To show the potential, broad applicability and challenges of DAC, we explore how a set of six initial benchmarks compare in several dimensions of difficulty.