# Automated Algorithm Selection: from Feature-Based to Feature-Free Approaches

arXiv: https://arxiv.org/abs/2203.13392 · HTML: https://mlsanity.com/paper/2203.13392/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 77 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [A Critical Review of Recurrent Neural Networks for Sequence Learning](https://mlsanity.com/paper/1506.00019/) (2015)
- [LSTM Fully Convolutional Networks for Time Series Classification](https://mlsanity.com/paper/1709.05206/) (2017)
- [Automated Algorithm Selection: Survey and Perspectives](https://mlsanity.com/paper/1811.11597/) (2018)
- [Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learning](https://mlsanity.com/paper/1711.08921/) (2017)
- [Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method](https://mlsanity.com/paper/1708.05930/) (2017)
- [Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks](https://mlsanity.com/paper/1603.03827/) (2016)
- [On the Origin of Deep Learning](https://mlsanity.com/paper/1702.07800/) (2017)
- [Deep Learning as a Competitive Feature-Free Approach for Automated Algorithm Selection on the Traveling Salesperson Problem](https://mlsanity.com/paper/2006.15968/) (2020)
- [Clustering Stream Data by Exploring the Evolution of Density Mountain](https://mlsanity.com/paper/1710.00867/) (2017)
- [Small Boxes Big Data: A Deep Learning Approach to Optimize Variable Sized Bin Packing](https://mlsanity.com/paper/1702.04415/) (2017)

## Built on by (most cited first, 13 of 13)

- [Large Language Model-Enhanced Algorithm Selection: Towards Comprehensive Algorithm Representation](https://mlsanity.com/paper/2311.13184/) (2023)
- [On the Utility of Probing Trajectories for Algorithm-Selection](https://mlsanity.com/paper/2401.12745/) (2024)
- [Revisit the Algorithm Selection Problem for TSP with Spatial Information Enhanced Graph Neural Networks](https://mlsanity.com/paper/2302.04035/) (2023)
- [Improving Algorithm-Selection and Performance-Prediction via Learning Discriminating Training Samples](https://mlsanity.com/paper/2404.05359/) (2024)
- [Identifying Easy Instances to Improve Efficiency of ML Pipelines for Algorithm-Selection](https://mlsanity.com/paper/2406.16999/) (2024)
- [Behavior and Representation in Open-Weight Large Language Models for Combinatorial Optimization: From Feature Extraction to Algorithm Selection](https://mlsanity.com/paper/2512.13374/) (2025)
- [Beyond Numerical Features: CNN-Driven Algorithm Selection via Contour Plots for Continuous Black-Box Optimization](https://mlsanity.com/paper/2605.20797/) (2026)
- [Evaluating the Robustness of Deep-Learning Algorithm-Selection Models by Evolving Adversarial Instances](https://mlsanity.com/paper/2406.16609/) (2024)
- [Automatic Feature Learning for Essence: a Case Study on Car Sequencing](https://mlsanity.com/paper/2409.15158/) (2024)
- [Algorithm Selection with Probing Trajectories: Benchmarking the Choice of Classifier Model](https://mlsanity.com/paper/2501.11414/) (2025)
- [Geometric Probing for Algorithm Selection in Continuous Black-Box Optimization](https://mlsanity.com/paper/2604.09095/) (2026)
- [Algorithm Selection with Zero Domain Knowledge via Text Embeddings](https://mlsanity.com/paper/2604.19753/) (2026)
- [On the Influence of the Feature Computation Budget on Per-Instance Algorithm Selection for Black-Box Optimization](https://mlsanity.com/paper/2605.04954/) (2026)

## Abstract (arXiv)

We propose a novel technique for algorithm-selection, applicable to optimisation domains in which there is implicit sequential information encapsulated in the data, e.g., in online bin-packing. Specifically we train two types of recurrent neural networks to predict a packing heuristic in online bin-packing, selecting from four well-known heuristics. As input, the RNN methods only use the sequence of item-sizes. This contrasts to typical approaches to algorithm-selection which require a model to be trained using domain-specific instance features that need to be first derived from the input data. The RNN approaches are shown to be capable of achieving within 5% of the oracle performance on between 80.88% to 97.63% of the instances, depending on the dataset. They are also shown to outperform classical machine learning models trained using derived features. Finally, we hypothesise that the proposed methods perform well when the instances exhibit some implicit structure that results in discriminatory performance with respect to a set of heuristics. We test this hypothesis by generating fourteen new datasets with increasing levels of structure, and show that there is a critical threshold of structure required before algorithm-selection delivers benefit.