# PS-AAS: Portfolio Selection for Automated Algorithm Selection in Black-Box Optimization

arXiv: https://arxiv.org/abs/2310.10685 · HTML: https://mlsanity.com/paper/2310.10685/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 14 Oct 2023.

## Builds on (most cited first, 15 of 15)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Automated Algorithm Selection: Survey and Perspectives](https://mlsanity.com/paper/1811.11597/) (2018)
- [Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learning](https://mlsanity.com/paper/1711.08921/) (2017)
- [Exploratory Landscape Analysis is Strongly Sensitive to the Sampling Strategy](https://mlsanity.com/paper/2006.11135/) (2020)
- [IOHanalyzer: Detailed Performance Analyses for Iterative Optimization Heuristics](https://mlsanity.com/paper/2007.03953/) (2020)
- [Tuning as a Means of Assessing the Benefits of New Ideas in Interplay with Existing Algorithmic Modules](https://mlsanity.com/paper/2102.12905/) (2021)
- [Per-run Algorithm Selection with Warm-starting using Trajectory-based Features](https://mlsanity.com/paper/2204.09483/) (2022)
- [Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking](https://mlsanity.com/paper/2010.04542/) (2020)
- [The Impact of Hyper-Parameter Tuning for Landscape-Aware Performance Regression and Algorithm Selection](https://mlsanity.com/paper/2104.09272/) (2021)
- [SELECTOR: Selecting a Representative Benchmark Suite for Reproducible Statistical Comparison](https://mlsanity.com/paper/2204.11527/) (2022)
- [Modular Differential Evolution](https://mlsanity.com/paper/2304.09524/) (2023)
- [OPTION: OPTImization Algorithm Benchmarking ONtology](https://mlsanity.com/paper/2104.11889/) (2021)
- [A Modular Hybridization of Particle Swarm Optimization and Differential Evolution](https://mlsanity.com/paper/2006.11886/) (2020)
- [The Importance of Landscape Features for Performance Prediction of Modular CMA-ES Variants](https://mlsanity.com/paper/2204.07431/) (2022)
- [Using Knowledge Graphs for Performance Prediction of Modular Optimization Algorithms](https://mlsanity.com/paper/2301.09876/) (2023)

## Built on by (most cited first, 5 of 5)

- [Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimization](https://mlsanity.com/paper/2411.00625/) (2024)
- [Tracing the Interactions of Modular CMA-ES Configurations Across Problem Landscapes](https://mlsanity.com/paper/2507.02331/) (2025)
- [Consistent and Distinctive: LLM Benchmark Efficiency via Maximum Independent Set Prompt Selection on Similarity Graphs](https://mlsanity.com/paper/2606.01400/) (2026)
- [On the Influence of the Feature Computation Budget on Per-Instance Algorithm Selection for Black-Box Optimization](https://mlsanity.com/paper/2605.04954/) (2026)
- [Partition Scores Are Not System Scores: Deployment-Fidelity Gaps in Decomposed Algorithm Selection](https://mlsanity.com/paper/2609.13785/) (2026)

## Abstract (arXiv)

The performance of automated algorithm selection (AAS) strongly depends on the portfolio of algorithms to choose from. Selecting the portfolio is a non-trivial task that requires balancing the trade-off between the higher flexibility of large portfolios with the increased complexity of the AAS task. In practice, probably the most common way to choose the algorithms for the portfolio is a greedy selection of the algorithms that perform well in some reference tasks of interest. We set out in this work to investigate alternative, data-driven portfolio selection techniques. Our proposed method creates algorithm behavior meta-representations, constructs a graph from a set of algorithms based on their meta-representation similarity, and applies a graph algorithm to select a final portfolio of diverse, representative, and non-redundant algorithms. We evaluate two distinct meta-representation techniques (SHAP and performance2vec) for selecting complementary portfolios from a total of 324 different variants of CMA-ES for the task of optimizing the BBOB single-objective problems in dimensionalities 5 and 30 with different cut-off budgets. We test two types of portfolios: one related to overall algorithm behavior and the `personalized' one (related to algorithm behavior per each problem separately). We observe that the approach built on the performance2vec-based representations favors small portfolios with negligible error in the AAS task relative to the virtual best solver from the selected portfolio, whereas the portfolios built from the SHAP-based representations gain from higher flexibility at the cost of decreased performance of the AAS. Across most considered scenarios, personalized portfolios yield comparable or slightly better performance than the classical greedy approach. They outperform the full portfolio in all scenarios.