mlsanityData to 14 Oct 2026

DACBench: A Benchmark Library for Dynamic Algorithm Configuration

arXiv 2105.08541 ↗First posted 18 May 2021cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Dynamic Algorithm Configuration (DAC) aims to dynamically control a target algorithm's hyperparameters in order to improve its performance. Several theoretical and empirical results have demonstrated the benefits of dynamically controlling hyperparameters in domains like evolutionary computation, AI Planning or deep learning. Replicating these results, as well as studying new methods for DAC, however, is difficult since existing benchmarks are often specialized and incompatible with the same interfaces. To facilitate benchmarking and thus research on DAC, we propose DACBench, a benchmark library that seeks to collect and standardize existing DAC benchmarks from different AI domains, as well as provide a template for new ones. For the design of DACBench, we focused on important desiderata, such as (i) flexibility, (ii) reproducibility, (iii) extensibility and (iv) automatic documentation and visualization. To show the potential, broad applicability and challenges of DAC, we explore how a set of six initial benchmarks compare in several dimensions of difficulty.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 14 of its 41 references that are in the map. First posted on arXiv 18 May 2021.

Builds on

14 of 41 references.

References · 14

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  2. 2016OpenAI Gymcited by 1,736
  3. 2016Learning to learn by gradient descent by gradient descentcited by 620
  4. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  5. 2015Contextual Markov Decision Processescited by 220
  6. 2016COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Settingcited by 138
  7. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 130
  8. 2014ParamILS: An Automatic Algorithm Configuration Frameworkcited by 89
  9. 2018Evolved Policy Gradientscited by 88
  10. 2015ASlib: A Benchmark Library for Algorithm Selectioncited by 37
  11. 2018IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristicscited by 28
  12. 2020Learning Heuristic Selection with Dynamic Algorithm Configurationcited by 21
2 more
  1. 2019Online Selection of CMA-ES Variantscited by 11
  2. 2015The Configurable SAT Solver Challenge (CSSC)cited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 3; 2024: 7; 2025: 8; 2026: 1.

28 papers.

Sort:

By area:

  1. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  2. 2022Automated Dynamic Algorithm Configurationcited by 39
  3. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  4. 2022A Survey of Methods for Automated Algorithm Configurationcited by 30
  5. 2024ConfigX: Modular Configuration for Evolutionary Algorithms via Multitask Reinforcement Learningcited by 15
  6. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  7. 2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learningcited by 13
  8. 2022Theory-inspired Parameter Control Benchmarks for Dynamic Algorithm Configurationcited by 11
  9. 2021EXPObench: Benchmarking Surrogate-based Optimisation Algorithms on Expensive Black-box Functionscited by 9
  10. 2023To Switch or not to Switch: Predicting the Benefit of Switching between Algorithms based on Trajectory Featurescited by 9
  11. 2024Hierarchical Transformers are Efficient Meta-Reinforcement Learnerscited by 8
  12. 2024Can we hop in general? A discussion of benchmark selection and design using the Hopper environmentcited by 7
16 more
  1. 2023Using Automated Algorithm Configuration for Parameter Controlcited by 5
  2. 2024Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalizationcited by 5
  3. 2024CALE: Continuous Arcade Learning Environmentcited by 5
  4. 2025On the Importance of Reward Design in Reinforcement Learning-based Dynamic Algorithm Configuration: A Case Study on OneMax with (1+(λ,λ))-GAcited by 5
  5. 2024CANDID DAC: Leveraging Coupled Action Dimensions with Importance Differences in DACcited by 4
  6. 2025Sequential Multi-Agent Dynamic Algorithm Configurationcited by 3
  7. 2021Automated Machine Learning, Bounded Rationality, and Rational Metareasoningcited by 2
  8. 2023Discovering Multiple Algorithm Configurationscited by 1
  9. 2025Deep Reinforcement Learning for Dynamic Algorithm Configuration: A Case Study on Optimizing OneMax with the (1+(λ,λ))-GAcited by 1
  10. 2026Large Language Models as Optimization Controllers: Adaptive Continuation for SIMP Topology Optimizationcited by 1
  11. 2022Explainable Model-specific Algorithm Selection for Multi-Label Classificationcited by 0
  12. 2024A Research Agenda for Usability and Generalisation in Reinforcement Learningcited by 0
  13. 2025Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimizationcited by 0
  14. 2025Revisiting Learning Rate Controlcited by 0
  15. 2025OPAL: Operator-Programmed Algorithms for Landscape-Aware Black-Box Optimizationcited by 0
  16. 2025Best Practices For Empirical Meta-Algorithmic Research: Guidelines from the COSEAL Research Networkcited by 0