mlsanityData to 14 Oct 2026

Pitfalls and Best Practices in Algorithm Configuration

arXiv 1705.06058 ↗First posted 17 May 2017cs.AI cs.SEcited by 25

Not on the map. Why?

Abstract (arXiv)

Good parameter settings are crucial to achieve high performance in many areas of artificial intelligence (AI), such as propositional satisfiability solving, AI planning, scheduling, and machine learning (in particular deep learning). Automated algorithm configuration methods have recently received much attention in the AI community since they replace tedious, irreproducible and error-prone manual parameter tuning and can lead to new state-of-the-art performance. However, practical applications of algorithm configuration are prone to several (often subtle) pitfalls in the experimental design that can render the procedure ineffective. We identify several common issues and propose best practices for avoiding them. As one possibility for automatically handling as many of these as possible, we also propose a tool called GenericWrapper4AC.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 2; 2024: 7; 2025: 5; 2026: 3.

25 papers.

Sort:

By area:

  1. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 145
  2. 2025MLGym: A New Framework and Benchmark for Advancing AI Research Agentscited by 124
  3. 2021HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPOcited by 112
  4. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  5. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 59
  6. 2022A Survey of Methods for Automated Algorithm Configurationcited by 30
  7. 2026Can LLMs Beat Classical Hyperparameter Optimization Algorithms? A Study on autoresearchcited by 15
  8. 2024Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimizationcited by 7
  9. 2023Taking the human out of decomposition-based optimization via artificial intelligence: Part II. Learning to initializecited by 5
  10. 2024Combining Automated Optimisation of Hyperparameters and Reward Shapecited by 5
  11. 2024A Tutorial on the Design, Experimentation and Application of Metaheuristic Algorithms to Real-World Optimization Problemscited by 5
  12. 2024Accelerating process control and optimization via machine learning: A reviewcited by 5
13 more
  1. 2025Overtuning in Hyperparameter Optimizationcited by 5
  2. 2022Assessing Ranking and Effectiveness of Evolutionary Algorithm Hyperparameters Using Global Sensitivity Analysis Methodologiescited by 3
  3. 2025carps: A Framework for Comparing N Hyperparameter Optimizers on M Benchmarkscited by 3
  4. 2024A learning-based mathematical programming formulation for the automatic configuration of optimization solverscited by 2
  5. 2024The Cross-environment Hyperparameter Setting Benchmark for Reinforcement Learningcited by 2
  6. 2026Performance Variation in Deep Reinforcement Learningcited by 2
  7. 2021An Automated Machine Learning (AutoML) Method for Driving Distraction Detection Based on Lane-Keeping Performancecited by 1
  8. 2022On the Configuration of More and Less Expressive Logic Programscited by 1
  9. 2024Learning to Configure Mathematical Programming Solvers by Mathematical Programmingcited by 1
  10. 2025Training neural networks faster with minimal tuning using pre-computed lists of hyperparameters for NAdamWcited by 1
  11. 2021(Machine) Learning to Improve the Empirical Performance of Discrete Algorithmscited by 0
  12. 2025Best Practices For Empirical Meta-Algorithmic Research: Guidelines from the COSEAL Research Networkcited by 0
  13. 2026Have I Solved This Before? Retrieving Similar Segmentation Problems for Evolutionary Learningcited by 0