# Learning Heuristic Selection with Dynamic Algorithm Configuration

arXiv: https://arxiv.org/abs/2006.08246 · HTML: https://mlsanity.com/paper/2006.08246/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2020.

## Built on by (most cited first, 21 of 21)

- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [Automated Dynamic Algorithm Configuration](https://mlsanity.com/paper/2205.13881/) (2022)
- [Multi-agent Dynamic Algorithm Configuration](https://mlsanity.com/paper/2210.06835/) (2022)
- [A Survey of Methods for Automated Algorithm Configuration](https://mlsanity.com/paper/2202.01651/) (2022)
- [DACBench: A Benchmark Library for Dynamic Algorithm Configuration](https://mlsanity.com/paper/2105.08541/) (2021)
- [Self-Paced Context Evaluation for Contextual Reinforcement Learning](https://mlsanity.com/paper/2106.05110/) (2021)
- [TransPath: Learning Heuristics For Grid-Based Pathfinding via Transformers](https://mlsanity.com/paper/2212.11730/) (2022)
- [Theory-inspired Parameter Control Benchmarks for Dynamic Algorithm Configuration](https://mlsanity.com/paper/2202.03259/) (2022)
- [PI is back! Switching Acquisition Functions in Bayesian Optimization](https://mlsanity.com/paper/2211.01455/) (2022)
- [Message Passing Variational Autoregressive Network for Solving Intractable Ising Models](https://mlsanity.com/paper/2404.06225/) (2024)
- [Using Automated Algorithm Configuration for Parameter Control](https://mlsanity.com/paper/2302.12334/) (2023)
- [Self-Adjusting Weighted Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2306.04262/) (2023)
- [Sequential Multi-Agent Dynamic Algorithm Configuration](https://mlsanity.com/paper/2510.23535/) (2025)
- [Automated Machine Learning, Bounded Rationality, and Rational Metareasoning](https://mlsanity.com/paper/2109.04744/) (2021)
- [Symbolic Search for Optimal Planning with Expressive Extensions](https://mlsanity.com/paper/2204.00288/) (2022)
- [A Training Data Recipe to Accelerate A* Search with Language Models](https://mlsanity.com/paper/2407.09985/) (2024)
- [Athanor: Local Search over Abstract Constraint Specifications](https://mlsanity.com/paper/2410.05937/) (2024)
- [Multi-parameter Control for the (1+(λ,λ))-GA on OneMax via Deep Reinforcement Learning](https://mlsanity.com/paper/2505.12982/) (2025)
- [Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization](https://mlsanity.com/paper/2505.16471/) (2025)
- [Observations Meet Actions: Learning Control-Sufficient Representations for Robust Policy Generalization](https://mlsanity.com/paper/2507.19437/) (2025)
- [Discovering Interpretable Multi-Parameter Control Policies for Evolutionary Algorithms Using Deep Reinforcement Learning](https://mlsanity.com/paper/2606.10129/) (2026)

## Abstract (arXiv)

A key challenge in satisficing planning is to use multiple heuristics within one heuristic search. An aggregation of multiple heuristic estimates, for example by taking the maximum, has the disadvantage that bad estimates of a single heuristic can negatively affect the whole search. Since the performance of a heuristic varies from instance to instance, approaches such as algorithm selection can be successfully applied. In addition, alternating between multiple heuristics during the search makes it possible to use all heuristics equally and improve performance. However, all these approaches ignore the internal search dynamics of a planning system, which can help to select the most useful heuristics for the current expansion step. We show that dynamic algorithm configuration can be used for dynamic heuristic selection which takes into account the internal search dynamics of a planning system. Furthermore, we prove that this approach generalizes over existing approaches and that it can exponentially improve the performance of the heuristic search. To learn dynamic heuristic selection, we propose an approach based on reinforcement learning and show empirically that domain-wise learned policies, which take the internal search dynamics of a planning system into account, can exceed existing approaches.