# Improving Algorithm-Selection and Performance-Prediction via Learning Discriminating Training Samples

arXiv: https://arxiv.org/abs/2404.05359 · HTML: https://mlsanity.com/paper/2404.05359/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 51 references that are in the map.
- First posted on arXiv 8 Apr 2024.

## Builds on (most cited first, 16 of 16)

- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [Automated Algorithm Selection: Survey and Perspectives](https://mlsanity.com/paper/1811.11597/) (2018)
- [Per-run Algorithm Selection with Warm-starting using Trajectory-based Features](https://mlsanity.com/paper/2204.09483/) (2022)
- [Towards Feature-Based Performance Regression Using Trajectory Data](https://mlsanity.com/paper/2102.05370/) (2021)
- [Towards Explainable Exploratory Landscape Analysis: Extreme Feature Selection for Classifying BBOB Functions](https://mlsanity.com/paper/2102.00736/) (2021)
- [Automated Algorithm Selection: from Feature-Based to Feature-Free Approaches](https://mlsanity.com/paper/2203.13392/) (2022)
- [Deep Learning as a Competitive Feature-Free Approach for Automated Algorithm Selection on the Traveling Salesperson Problem](https://mlsanity.com/paper/2006.15968/) (2020)
- [To Switch or not to Switch: Predicting the Benefit of Switching between Algorithms based on Trajectory Features](https://mlsanity.com/paper/2302.09075/) (2023)
- [DynamoRep: Trajectory-Based Population Dynamics for Classification of Black-box Optimization Problems](https://mlsanity.com/paper/2306.05438/) (2023)
- [On the Utility of Probing Trajectories for Algorithm-Selection](https://mlsanity.com/paper/2401.12745/) (2024)
- [Benchmarking Feature-based Algorithm Selection Systems for Black-box Numerical Optimization](https://mlsanity.com/paper/2109.08377/) (2021)
- [Explorative Data Analysis of Time Series based AlgorithmFeatures of CMA-ES Variants](https://mlsanity.com/paper/2104.08098/) (2021)
- [Can Genetic Programming Do Manifold Learning Too?](https://mlsanity.com/paper/1902.02949/) (2019)
- [Quality-Diversity Meta-Evolution: customising behaviour spaces to a meta-objective](https://mlsanity.com/paper/2109.03918/) (2021)
- [Automated Algorithm Selection for Radar Network Configuration](https://mlsanity.com/paper/2205.03670/) (2022)

## Built on by (most cited first, 2 of 2)

- [Identifying Easy Instances to Improve Efficiency of ML Pipelines for Algorithm-Selection](https://mlsanity.com/paper/2406.16999/) (2024)
- [Algorithm Selection with Probing Trajectories: Benchmarking the Choice of Classifier Model](https://mlsanity.com/paper/2501.11414/) (2025)

## Abstract (arXiv)

The choice of input-data used to train algorithm-selection models is recognised as being a critical part of the model success. Recently, feature-free methods for algorithm-selection that use short trajectories obtained from running a solver as input have shown promise. However, it is unclear to what extent these trajectories reliably discriminate between solvers. We propose a meta approach to generating discriminatory trajectories with respect to a portfolio of solvers. The algorithm-configuration tool irace is used to tune the parameters of a simple Simulated Annealing algorithm (SA) to produce trajectories that maximise the performance metrics of ML models trained on this data. We show that when the trajectories obtained from the tuned SA algorithm are used in ML models for algorithm-selection and performance prediction, we obtain significantly improved performance metrics compared to models trained both on raw trajectory data and on exploratory landscape features.