# The Algorithm Selection Competitions 2015 and 2017

arXiv: https://arxiv.org/abs/1805.01214 · HTML: https://mlsanity.com/paper/1805.01214/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2018.

## Built on by (most cited first, 12 of 12)

- [Algorithm Selection on a Meta Level](https://mlsanity.com/paper/2107.09414/) (2021)
- [Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classification](https://mlsanity.com/paper/2206.08138/) (2022)
- [CMA-ES for Post Hoc Ensembling in AutoML: A Great Success and Salvageable Failure](https://mlsanity.com/paper/2307.00286/) (2023)
- [Q(D)O-ES: Population-based Quality (Diversity) Optimisation for Post Hoc Ensemble Selection in AutoML](https://mlsanity.com/paper/2307.08364/) (2023)
- [Advances in MetaDL: AAAI 2021 challenge and workshop](https://mlsanity.com/paper/2202.01890/) (2022)
- [On the evaluation of (meta-)solver approaches](https://mlsanity.com/paper/2202.08613/) (2022)
- [Automatic Algorithm Selection for Pseudo-Boolean Optimization with Given Computational Time Limits](https://mlsanity.com/paper/2309.03924/) (2023)
- [Characterising harmful data sources when constructing multi-fidelity surrogate models](https://mlsanity.com/paper/2403.08118/) (2024)
- [On Constructing Algorithm Portfolios in Algorithm Selection for Computationally Expensive Black-box Optimization in the Fixed-budget Setting](https://mlsanity.com/paper/2405.10976/) (2024)
- [Algorithm Selection with Zero Domain Knowledge via Text Embeddings](https://mlsanity.com/paper/2604.19753/) (2026)
- [Partition Scores Are Not System Scores: Deployment-Fidelity Gaps in Decomposed Algorithm Selection](https://mlsanity.com/paper/2609.13785/) (2026)
- [Certify the Decision, Not Every Score: Exact Top-k Selection from Few Evaluations](https://mlsanity.com/paper/2610.14996/) (2026)

## Abstract (arXiv)

The algorithm selection problem is to choose the most suitable algorithm for solving a given problem instance. It leverages the complementarity between different approaches that is present in many areas of AI. We report on the state of the art in algorithm selection, as defined by the Algorithm Selection competitions in 2015 and 2017. The results of these competitions show how the state of the art improved over the years. We show that although performance in some cases is very good, there is still room for improvement in other cases. Finally, we provide insights into why some scenarios are hard, and pose challenges to the community on how to advance the current state of the art.