# ASlib: A Benchmark Library for Algorithm Selection

arXiv: https://arxiv.org/abs/1506.02465 · HTML: https://mlsanity.com/paper/1506.02465/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2015.

## Built on by (most cited first, 30 of 37)

- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization](https://mlsanity.com/paper/2109.03670/) (2021)
- [Automated Dynamic Algorithm Configuration](https://mlsanity.com/paper/2205.13881/) (2022)
- [PolyNet: Learning Diverse Solution Strategies for Neural Combinatorial Optimization](https://mlsanity.com/paper/2402.14048/) (2024)
- [DACBench: A Benchmark Library for Dynamic Algorithm Configuration](https://mlsanity.com/paper/2105.08541/) (2021)
- [Large Language Model-Enhanced Algorithm Selection: Towards Comprehensive Algorithm Representation](https://mlsanity.com/paper/2311.13184/) (2023)
- [Statistical Comparisons of Classifiers by Generalized Stochastic Dominance](https://mlsanity.com/paper/2209.01857/) (2022)
- [Zero-Shot AutoML with Pretrained Models](https://mlsanity.com/paper/2206.08476/) (2022)
- [Algorithm Selection on a Meta Level](https://mlsanity.com/paper/2107.09414/) (2021)
- [Artificial Intelligence for Operations Research: Revolutionizing the Operations Research Process](https://mlsanity.com/paper/2401.03244/) (2024)
- [Statistical Multicriteria Benchmarking via the GSD-Front](https://mlsanity.com/paper/2406.03924/) (2024)
- [Learning How to Optimize Black-Box Functions With Extreme Limits on the Number of Function Evaluations](https://mlsanity.com/paper/2103.10321/) (2021)
- [Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization](https://mlsanity.com/paper/2407.13513/) (2024)
- [Optimizing with Low Budgets: a Comparison on the Black-box Optimization Benchmarking Suite and OpenAI Gym](https://mlsanity.com/paper/2310.00077/) (2023)
- [Machine Learning for Online Algorithm Selection under Censored Feedback](https://mlsanity.com/paper/2109.06234/) (2021)
- [HARRIS: Hybrid Ranking and Regression Forests for Algorithm Selection](https://mlsanity.com/paper/2210.17341/) (2022)
- [Comprehensive Algorithm Portfolio Evaluation using Item Response Theory](https://mlsanity.com/paper/2307.15850/) (2023)
- [Global Benchmark Database](https://mlsanity.com/paper/2405.10045/) (2024)
- [Evaluating Meta-Feature Selection for the Algorithm Recommendation Problem](https://mlsanity.com/paper/2106.03954/) (2021)
- [Constructing a meta-learner for unsupervised anomaly detection](https://mlsanity.com/paper/2304.11438/) (2023)
- [Does AI for science need another ImageNet Or totally different benchmarks? A case study of machine learning force fields](https://mlsanity.com/paper/2308.05999/) (2023)
- [Context-Aware Synthesis of Optimization Pipelines for Warehouse Optimization](https://mlsanity.com/paper/2606.26852/) (2026)
- [(Machine) Learning to Improve the Empirical Performance of Discrete Algorithms](https://mlsanity.com/paper/2109.14271/) (2021)
- [Predictive Machine Learning of Objective Boundaries for Solving COPs](https://mlsanity.com/paper/2111.03160/) (2021)
- [On the evaluation of (meta-)solver approaches](https://mlsanity.com/paper/2202.08613/) (2022)
- [Explainable Model-specific Algorithm Selection for Multi-Label Classification](https://mlsanity.com/paper/2211.11227/) (2022)
- [Automatic Algorithm Selection for Pseudo-Boolean Optimization with Given Computational Time Limits](https://mlsanity.com/paper/2309.03924/) (2023)
- [Characterising harmful data sources when constructing multi-fidelity surrogate models](https://mlsanity.com/paper/2403.08118/) (2024)
- [Frugal Algorithm Selection](https://mlsanity.com/paper/2405.11059/) (2024)
- [Utilizing Data Fingerprints for Privacy-Preserving Algorithm Selection in Time Series Classification: Performance and Uncertainty Estimation on Unseen Datasets](https://mlsanity.com/paper/2409.08636/) (2024)

## Abstract (arXiv)

The task of algorithm selection involves choosing an algorithm from a set of algorithms on a per-instance basis in order to exploit the varying performance of algorithms over a set of instances. The algorithm selection problem is attracting increasing attention from researchers and practitioners in AI. Years of fruitful applications in a number of domains have resulted in a large amount of data, but the community lacks a standard format or repository for this data. This situation makes it difficult to share and compare different approaches effectively, as is done in other, more established fields. It also unnecessarily hinders new researchers who want to work in this area. To address this problem, we introduce a standardized format for representing algorithm selection scenarios and a repository that contains a growing number of data sets from the literature. Our format has been designed to be able to express a wide variety of different scenarios. Demonstrating the breadth and power of our platform, we describe a set of example experiments that build and evaluate algorithm selection models through a common interface. The results display the potential of algorithm selection to achieve significant performance improvements across a broad range of problems and algorithms.