# Exploring Opportunistic Meta-knowledge to Reduce Search Spaces for Automated Machine Learning

arXiv: https://arxiv.org/abs/2105.00282 · HTML: https://mlsanity.com/paper/2105.00282/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 1 May 2021.

## Builds on (most cited first, 9 of 9)

- [Tunability: Importance of Hyperparameters of Machine Learning Algorithms](https://mlsanity.com/paper/1802.09596/) (2018)
- [Hyperparameter Importance Across Datasets](https://mlsanity.com/paper/1710.04725/) (2017)
- [Importance of Tuning Hyperparameters of Machine Learning Algorithms](https://mlsanity.com/paper/2007.07588/) (2020)
- [Benchmark and Survey of Automated Machine Learning Frameworks](https://mlsanity.com/paper/1904.12054/) (2019)
- [AutonoML: Towards an Integrated Framework for Autonomous Machine Learning](https://mlsanity.com/paper/2012.12600/) (2020)
- [Automatic Composition and Optimization of Multicomponent Predictive Systems With an Extended Auto-WEKA](https://mlsanity.com/paper/1612.08789/) (2016)
- [A Review of Meta-level Learning in the Context of Multi-component, Multi-level Evolving Prediction Systems](https://mlsanity.com/paper/2007.10818/) (2020)
- [AVATAR -- Machine Learning Pipeline Evaluation Using Surrogate Model](https://mlsanity.com/paper/2001.11158/) (2020)
- [AutoWeka4MCPS-AVATAR: Accelerating Automated Machine Learning Pipeline Composition and Optimisation](https://mlsanity.com/paper/2011.11846/) (2020)

## Built on by (most cited first, 2 of 2)

- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [On Taking Advantage of Opportunistic Meta-knowledge to Reduce Configuration Spaces for Automated Machine Learning](https://mlsanity.com/paper/2208.04376/) (2022)

## Abstract (arXiv)

Machine learning (ML) pipeline composition and optimisation have been studied to seek multi-stage ML models, i.e. preprocessor-inclusive, that are both valid and well-performing. These processes typically require the design and traversal of complex configuration spaces consisting of not just individual ML components and their hyperparameters, but also higher-level pipeline structures that link these components together. Optimisation efficiency and resulting ML-model accuracy both suffer if this pipeline search space is unwieldy and excessively large; it becomes an appealing notion to avoid costly evaluations of poorly performing ML components ahead of time. Accordingly, this paper investigates whether, based on previous experience, a pool of available classifiers/regressors can be preemptively culled ahead of initiating a pipeline composition/optimisation process for a new ML problem, i.e. dataset. The previous experience comes in the form of classifier/regressor accuracy rankings derived, with loose assumptions, from a substantial but non-exhaustive number of pipeline evaluations; this meta-knowledge is considered 'opportunistic'. Numerous experiments with the AutoWeka4MCPS package, including ones leveraging similarities between datasets via the relative landmarking method, show that, despite its seeming unreliability, opportunistic meta-knowledge can improve ML outcomes. However, results also indicate that the culling of classifiers/regressors should not be too severe either. In effect, it is better to search through a 'top tier' of recommended predictors than to pin hopes onto one previously supreme performer.